Skip to content
All articles
LLMs1 min read

Local LLMs on a DevOps Workstation

Practical notes on Ollama, quantization, and when local models beat cloud APIs for infrastructure work.

ollamalocal-llmprivacy

Running models locally changes the economics of iteration. You can try prompts against sensitive logs without sending them to a third party.

What works well locally

  • Drafting kubectl / Terraform snippets
  • Summarizing CI logs
  • Rewriting runbook sections
  • Exploring MCP tool schemas

What still needs frontier models

  • Long multi-repo reasoning
  • High-stakes security review
  • Complex multi-agent orchestration
ollama pull llama3.2
ollama run llama3.2

Start local, escalate to cloud when quality or context demands it.