Million-Token Context Windows Are Here: Which Workflows Actually Benefit

The change is real: million-token context is now exposed in IDE agents and mainstream model APIs, not just in a model-card footnote. But a large window is capacity, not automatic understanding. It helps when the answer depends on relationships spread across many modules, commits, logs, or documents; it is a poor substitute for a narrow task, a test command, and a reviewer who can tell whether the resulting diff belongs in the codebase.

GitHub Copilot makes 1M context a selectable IDE and CLI option

GitHub Copilot now documents an extended 1 million-token context option for supported models in VS Code and Copilot CLI. It is not the default: GitHub explicitly recommends regular context and reasoning for ordinary work because the larger window consumes more AI credits, then reserving the extended setting for large codebases, long documents, and complex multi-file work.

The practical use is a task such as tracing an auth migration through service code, infrastructure, tests, and runbooks without manually attaching a sequence of files. It is not useful for “fix this failing unit test”: feeding a million tokens to that job mostly gives the agent more irrelevant ways to be wrong.

Why it matters: if Copilot is already in your editor or terminal, the first experiment is operational rather than architectural—select an extended-context model for one genuinely cross-repository investigation, then compare the patch and credit usage against your usual scoped prompt.

OpenAI’s current API models put 1.05M tokens behind a price threshold

OpenAI’s GPT-5.6 Sol, Terra, and Luna API models list a 1.05 million-token context window and a 128,000-token maximum output. The model pages also make the cost boundary concrete: for Luna, prompts above 272,000 input tokens are billed at 2× the input rate and 1.5× the output rate for the entire request.

That makes API-built developer tooling a better fit than one-off chat for long context. A release-review bot can place a diff, changed API contracts, generated client changes, deployment manifests, and incident notes into one bounded job; it can then produce a checklist or an annotated risk report rather than pretending it can safely author an unrestricted rewrite.

Why it matters: measure the assembled prompt before sending it and put the expensive mode behind an explicit flag—say, --full-release-context—rather than silently making every routine agent turn a long-context request.

Gemini’s 1M window is especially useful for mixed code and operational artifacts

Google’s Gemini documentation describes 1 million tokens as roughly 50,000 lines of 80-character code, and current Gemini 3-series documentation lists 1M-token input windows on several models. The more interesting part for developers is multimodal input: a single investigation can include a repository slice, architecture PDFs, screenshots from a broken admin flow, and a long trace without first converting everything into plain text.

Google also warns that long-context retrieval degrades when a prompt contains multiple relevant “needles,” and that results vary by context. So use this for reconstruction work—“explain why these logs, this config change, and this design doc disagree”—not as proof that the model has exhaustively found every call site or authorization boundary.

Why it matters: long context is a credible replacement for the folder of hand-selected attachments during incident analysis, but it does not remove the need for rg, static analysis, targeted tests, and links back to the source artifacts.

Long windows have not removed compaction or context hygiene

Agent loops can burn context on their own output: shell transcripts, tool schemas, failed edits, test logs, and prior messages. OpenAI’s Codex engineering write-up says it compacts a conversation after it crosses a threshold, replacing the accumulated input with a smaller representation; GitHub similarly documents /compact in Copilot CLI and /context for checking current usage.

The workflow that benefits is a long-running implementation where the objective stays stable for hours: fix a migration, keep the decision record, and compact around the subsystem when the transcript grows. The workflow that does not is a chat that switches from an OAuth bug to a Kubernetes question to a code review; start a new session instead of asking the model to preserve accidental history.

Why it matters: keep AGENTS.md or equivalent instructions concise and repository-specific, restrict the tools exposed to the task, and use /compact focus on the auth module as maintenance—not as an emergency repair after the agent has lost the plot.

The economics favor reusable evidence, not repeated repository dumps

Million-token context is affordable only when the input is doing useful work repeatedly. Google offers context caching on paid Gemini API tiers, while Anthropic’s published 1M-context pricing for Claude Sonnet 4 applies premium rates once a request exceeds 200,000 input tokens: $6 per million input tokens and $22.50 per million output tokens, with the entire request charged at that tier.

The good candidate is stable material that supports multiple questions: an API specification, a generated SDK, a design corpus, or a fixed release bundle. The bad candidate is sending the whole monorepo because the request was underspecified; caching a bad prompt only makes an inefficient workflow cheaper to repeat.

Why it matters: add token counts and cache-hit metrics to the agent’s logs before optimizing model choice. A developer who can see “this review sent 310K uncached tokens” has a usable cost-control signal; a developer who only sees a surprisingly large monthly bill does not.