An Air-Gapped Coding Agent Is Mostly a Packaging Problem

Yes: an air-gapped coding agent can be useful for implementation, tests, small refactors, and repo navigation. The practical default is a local model server plus a terminal agent such as Aider—not an IDE extension with a pile of opaque cloud-adjacent features—and the hard part is packaging every dependency before you close the network door.

Treat “air-gapped” as an operational property you prove, not a checkbox in an agent’s settings. If the model server, the agent, the editor, a package installer, telemetry, documentation assets, or a browser tool still expects egress, you have an on-prem setup with a future outage built in.

Start with a narrow, boring architecture

Use three processes with obvious boundaries: a model server on a GPU host, an agent on the developer workstation or build runner, and the repository in a disposable worktree. vLLM is a reasonable server-side choice when you have compatible GPU infrastructure because it exposes an OpenAI-compatible API; Aider can target any OpenAI-compatible endpoint. That gives you a replaceable seam between the agent and whatever local model you approve. [1] [2]

For a first deployment, keep the inference endpoint on loopback or a private service network. Do not expose it broadly just because it accepts an API key. vLLM’s own documentation warns that its API-key setting does not authenticate every endpoint, including an inference-capable endpoint, and recommends network hardening such as a reverse proxy. [1]

# All packages, image layers, model files, tokenizer files, and chat template
# must already exist inside the disconnected environment.
vllm serve /srv/models/approved-coder-model \
  --host 127.0.0.1 \
  --port 8000 \
  --api-key "$LOCAL_LLM_TOKEN" \
  --enable-offline-docs

Passing a local model directory matters. It removes the convenient but unacceptable assumption that a server may resolve a model identifier from a public registry at startup. vLLM documents local-directory model paths, and its offline-docs flag avoids another small but annoying internet dependency in the server UI. [3] [4]

Use the agent from a clean Git worktree

Aider is a good first harness here because it is transparent about files, diffs, and commits. It supports local Ollama models and generic OpenAI-compatible servers; it also commits its changes and provides /undo. That is more useful in a restricted environment than a polished chat panel that quietly owns a session database, extension update path, and web-search integration. [2] [5]

export OPENAI_API_BASE=http://127.0.0.1:8000/v1
export OPENAI_API_KEY="$LOCAL_LLM_TOKEN"

cd /work/payments-service
git worktree add ../payments-agent -b agent/retry-fix
cd ../payments-agent

aider --model openai/approved-coder-model

Give the agent a task with a testable stop condition: “Add retry handling to this client; change only src/client.py and its tests; run pytest tests/client.” Then inspect the diff and run the test yourself. Don’t start with “understand this monorepo and improve it.” Local models have less margin for a sprawling context, and agent loops turn weak retrieval into a lot of confident shell activity.

Build an import bundle, not an installation guide

The day-one bundle should be versioned and checksummed in the connected build environment, then transferred through your approved media process. Make it a release artifact with a manifest, not a wiki page full of curl | sh commands.

• Pinned OS packages or a local package repository, plus GPU drivers and the exact runtime version.
• OCI images for the model server and sandbox, exported with docker save or your organization’s equivalent.
• Model weights, tokenizer, model configuration, and chat template together. A model file without its tokenizer or template is a surprisingly efficient way to get plausible nonsense.
• The agent wheel or binary and every Python or Node dependency in a local wheelhouse or package cache.
• The editor extension VSIX, if developers need one; disable automatic extension updates before the machine is disconnected.
• A model-and-agent acceptance test: one prompt, an expected changed file, and a command that must pass.

Run that acceptance test after import and before a developer depends on the system. Also test a deliberate failure: block all outbound traffic at the host and network layers, invoke the agent, and confirm that it still completes without DNS lookups, package downloads, or calls to vendor endpoints. “The prompt stayed private” is not enough if the machine still fetches a model card or update manifest.

Context size is where local deployments quietly fail

Don’t copy a cloud-agent workflow and assume the local endpoint will retain the same context. Aider specifically warns that Ollama’s default context can be only 2k tokens and that excess context may be silently discarded; its Ollama guide shows starting the server with an 8k context window and configuring a fixed num_ctx value where needed. Silent truncation is worse than a hard error: the agent still produces a diff, just without the constraints you thought it read. [6]

Make context visible in the prompt and workflow. Add only the files you need, use repo instructions that name the test command and forbidden directories, and start a new task when the work changes. Aider recommends adding only files relevant to the task because excess files can overwhelm the model; its /clear and /drop controls are worth using when a run starts repeating itself. [5] [7]

Keep tools less powerful than the model’s confidence

The dangerous part of an on-prem agent is not that it might disclose code to a SaaS API. It is that it can run commands beside production-like credentials, internal package registries, source mirrors, and deployment tooling. A disconnected network does not turn rm, git push, database clients, or CI credentials into safe tools.

Run the agent as an unprivileged user in a worktree with a minimal PATH. Mount source read-write only if you want edits; mount secrets never; keep SSH agents, cloud credentials, and production kubeconfigs outside the agent’s process. Start with read access plus explicit approval for shell commands and edits. If you later put the agent behind an internal service, use a sandbox container with an allowlisted command surface rather than handing it the host Docker socket.

This is also why local does not automatically mean autonomous. The local model may be slower and less reliable at structured edits than the cloud model you are used to. Aider’s documentation is blunt that lower-capability models can fail to return usable edit formats, and that quantized local models can have more editing problems. Plan for smaller changes, tests after every edit, and a human reviewing each commit. [7]

Measure the workflow you actually intend to permit

Before buying another GPU or rolling this out to a regulated team, record three numbers from your acceptance suite: time to first token, tokens per second during generation, and elapsed time from prompt to a passing test. Record failures separately: malformed edits, commands that needed approval, test failures, and context resets. Those are the numbers that determine whether developers will use the tool for a five-file bug fix or abandon it after lunch.

Use a small internal task set drawn from your own repositories: a dependency bump with tests, a failing-unit-test diagnosis, a two-file API change, and a refactor that must preserve a golden test. Pin the model artifact, quantization, serving flags, agent version, and prompt for every run. Otherwise, “the air-gapped agent got worse” will leave you comparing a different model, a different template, a different context limit, and a different harness all at once.

The resulting setup is less magical than a cloud coding agent. It won’t browse current docs, pull a missing package, inspect an external issue, or use a hosted model as a rescue path—and that is the point. Make the offline path explicit, make its artifacts reproducible, and keep its shell permissions narrow. Once that works, you have something an engineer can run before coffee without first negotiating an exception to the network policy.