Run an AI Coding Agent With Ollama and Local LLMs

To run an AI coding agent with Ollama and local LLMs, pull a coding model, make sure Ollama is serving it on localhost:11434, set a context window large enough for agent work, and launch or configure an agent against that local endpoint. The shortest supported path is ollama pull qwen3-coder followed by ollama launch opencode --model qwen3-coder; for any other agent, point its Ollama integration at the local server or its OpenAI-compatible base URL, http://localhost:11434/v1.

Run an AI coding agent with Ollama: the fast path

Start with the path that has the fewest moving parts. Ollama’s CLI can configure and launch supported integrations interactively, including OpenCode, Claude Code, Codex, VS Code, and Droid. For a local coding session, pull the model first so the agent does not discover a multi-gigabyte download halfway through its first task:

ollama pull qwen3-coder
ollama launch opencode --model qwen3-coder

That gets you an agent harness plus a local model, not merely a chat prompt in a terminal. The important difference is the loop: the harness gives the model files, tool schemas, command output, and follow-up turns; the model decides whether to read, edit, test, or stop. If you prefer a different harness, do not fake an OpenAI API key just to satisfy a settings form. Ollama documents both its native local API at http://localhost:11434/api and its OpenAI-compatible endpoint at http://localhost:11434/v1; local requests do not require authorization.

Before you hand the agent a real repository, give it a task with a cheap, binary result: “find the failing test, explain the failure, make the smallest fix, then run only that test.” You are testing tool use, edit format, command execution, and recovery from a failing test. A model that writes a plausible patch but repeatedly loses its place after command output is not ready for a migration or a broad refactor.

Set Ollama context length for coding agents

The default context is a frequent reason a local agent feels competent for one file and useless for a repository. Ollama’s current defaults vary by VRAM: under 24 GiB gets 4K tokens, 24–48 GiB gets 32K, and 48 GiB or more gets 256K. Ollama specifically recommends at least 64,000 tokens for agents and coding tools. Four thousand tokens can disappear into a system prompt, a directory listing, a few source files, and test output before the model has enough room to reason about the requested change.

In the Ollama desktop app, set the context-length slider to 64K or higher. When running the server yourself, start it with:

OLLAMA_CONTEXT_LENGTH=64000 ollama serve

Do this before starting the agent. If Ollama is already running as a desktop app or system service, changing a shell variable in a new terminal will not reconfigure that existing process; set the variable where the service is launched, then restart it. Confirm what actually loaded rather than trusting the setting:

ollama ps

Look at both CONTEXT and PROCESSOR. A large context consumes memory, and a model partly offloaded to CPU may turn every tool loop into a wait. This is the local-agent trade: context, model size, and responsiveness all compete for the same VRAM or unified memory. Start at 64K, verify that the model remains acceptably accelerated, and only then try a larger model or context.

Which local LLM should power a coding agent?

Pick for tool calling first, code quality second, and raw parameter count third. An agent asks the model to emit structured requests for actions such as reading a file, searching the tree, applying an edit, or executing a test. Ollama supports function calling, including multi-turn agent loops, but model support is still a model capability rather than a property magically added by the server. Check the model’s advertised capabilities before wiring it into an autonomous workflow.

qwen3-coder is a sensible starting point because Ollama lists it among its recommended local coding models. It is a starting point, not a universal answer. A smaller model that reliably requests the right file and runs the exact test can outperform a larger one that produces malformed tool arguments, makes speculative edits, or burns half the context explaining itself.

Use one repeatable evaluation task across models: an issue with an existing failing test, a clear success condition, and a small expected diff. Record four things: whether the agent found the right code, whether it chose the right commands, whether it preserved test intent, and how long a full tool loop took on your hardware. Do not compare one model’s response text with another’s. Compare the resulting diff and the command transcript.

How to connect another coding agent to Ollama

For an agent with a native Ollama provider, select that provider and use the model name returned by ollama ls. The usual local address is http://localhost:11434. For an agent that only offers OpenAI-compatible settings, use http://localhost:11434/v1 as its base URL and select the same locally pulled model. The latter is useful, but compatibility is an interface contract, not proof that every agent feature will work with every local model. Test file edits and shell-tool calls before enabling browser tools, MCP servers, or auto-approval.

Keep the server local unless you have a concrete reason not to. Ollama binds to 127.0.0.1:11434 by default. Changing OLLAMA_HOST to 0.0.0.0:11434 makes the service reachable on your network; do not do that casually. A coding-agent endpoint is more sensitive than a normal local chat endpoint because its client may send source code, command output, secrets exposed by test fixtures, and tool results. If a remote machine needs access, put authentication and network controls in front of it rather than publishing port 11434 directly.

Why local coding agents fail after the demo

The first failure mode is running a model that fits only by spilling across CPU and GPU. It will still answer, but the cost is multiplied by each read-command-write-test turn. The second is undersizing context: the agent starts well, then loses constraints from earlier files and quietly duplicates logic. The third is treating a local model as a privacy control while attaching tools that send data elsewhere. Local inference keeps prompts at the local server, but an MCP server, browser automation, package install, or remote test runner can independently leave the machine.

The fourth is over-automation. Do not begin with unrestricted shell approval. Give the agent a disposable worktree or branch, allow read-only inspection and narrowly scoped test commands, and inspect the diff after every meaningful step. Local models can be cheaper to iterate with, but they are not inherently better at recognizing a destructive command or an incorrect migration. The approval boundary belongs to the harness and your operating rules, not the model name.

How to keep a local agent responsive

Ollama keeps models in memory for five minutes by default. That is useful during an active debugging session because the next task avoids a reload; ollama ps shows what remains loaded. If memory is tight, release it with ollama stop <model>. If you intentionally want a model warm for repeated agent runs, Ollama also supports configuring keep-alive behavior through its API or server environment.

Avoid running two independent agent sessions on a machine that only fits one model at the chosen context. Ollama can queue requests when memory is insufficient, and parallel work also increases context-memory requirements. In practice, one agent with a model fully on the accelerator is usually more useful than two agents that alternate between stalled tool loops. Measure your own edit-test loop before adding concurrency.

Use Cline when you want a local model inside the editor

If the missing piece is an editor-native harness rather than another local model runner, Cline is an open-source coding agent available in VS Code and JetBrains Early Access. Its site says it can make coordinated project edits, run terminal commands, show checkpoints and diffs, and work with local Ollama or LM Studio as well as OpenAI-compatible endpoints. That makes it relevant here: keep Ollama as the local inference server, while using an agent interface that can plan work, apply changes, and let you approve the tool calls.

For the local route, Cline’s free open-source setup supports bring-your-own provider keys, endpoints, or weights, so a local Ollama model does not require a bundled per-seat model subscription. Cline also offers a separate ClinePass subscription for hosted open-weight models at $9.99 per month, with additional processing fees noted on its pricing page; that option is separate from pointing Cline at Ollama on your own machine. Use the local configuration when model control, offline-capable development, or avoiding provider API usage is the reason you are doing this in the first place.