Inference Prices Keep Falling: What the Latest Round of Cuts Means for Small Teams

Inference is getting cheaper mostly by giving teams more usable rungs on the model ladder, not by making one premium agent free to run forever. As of August 13, 2026, the relevant shift is that a small team can put classification, first-pass code review, test failure triage, documentation extraction, and batch migrations on models priced in cents per million tokens—then reserve expensive reasoning runs for changes a human will seriously review. The catch: tool calls, giant contexts, retries, and uncontrolled agent loops can still overwhelm a token-price win.

OpenAI shipped GPT-5.6 with a cheaper middle tier and a $1/$6 low-cost tier

OpenAI released the GPT-5.6 family on July 9, 2026. Its API list prices are $5 input / $30 output per million tokens for Sol, $2.50 / $15 for Terra, and $1 / $6 for Luna; OpenAI says Terra is priced at half of GPT-5.5 and positions Luna for cost-sensitive, high-volume work. The family also retains a 90% discount for cached reads, while cache writes cost 1.25× uncached input rates and have a 30-minute minimum cache life.

For a rough budget envelope, a run consuming 10 million input and 2 million output tokens is $22 on Luna before tools and retries. That is cheap enough to make a low-stakes queue of repository summaries or issue-label suggestions reasonable, but it is not evidence that Luna will replace a stronger model on an ambiguous refactor or security-sensitive review.

Why it matters: put the model name behind a routing rule rather than a product-wide default—start with Luna or Terra for repeatable work, promote only failed or high-risk tasks to Sol, and log both the route and the eventual human outcome. [OpenAI GPT-5.6 announcement; OpenAI API pricing]

Anthropic held token prices while moving more capability into Sonnet and Opus

This was not a nominal price cut, but it is part of the price-per-useful-result story. Anthropic released Sonnet 4.6 at the existing $3 input / $15 output per-million-token rate, then shipped Opus 4.6, 4.7, and 4.8 at $5 / $25; the models add features such as effort controls, context compaction, and longer-context options rather than asking developers to pay a new top-line token rate.

The bill has sharp edges. Prompts above 200,000 tokens on the Claude Platform carry premium pricing for Opus, and US-only inference is listed at 1.1× token pricing. Anthropic’s batch rate halves the standard rates—Sonnet 4.6 is $1.50 / $7.50 and Opus is $2.50 / $12.50 globally—but a batch job is the wrong execution path for an interactive IDE loop or a deploy-blocking CI check.

Why it matters: if Sonnet now clears your acceptance tests, moving routine agent work down from Opus is a much more reliable saving than trying to micromanage prompt wording. Keep an explicit escalation path for hard debugging and reviews; “same price” does not mean the expensive model has become the right default. [Anthropic Sonnet 4.6 announcement; Anthropic pricing list]

Gemini 3.1 Flash-Lite pushes bulk text work into the single-digit-dollar range

Google’s current Gemini 3.1 Flash-Lite preview lists $0.25 per million input tokens and $1.50 per million output tokens, with a 1 million-token context window and 64,000-token maximum output. Its Batch API is priced at half the interactive rate, so the same 10-million-input, 2-million-output example is $5.50 interactively or $2.75 in batch, before any paid search grounding or other add-ons.

Those numbers are compelling for jobs where you can validate the result mechanically: extracting API changes from changelogs, converting test output into structured failure records, or producing candidate documentation diffs. But it remains a preview model, and the pricing page separately charges for some tools; grounding with Google Search is not free after the included allowance, so an agent that searches liberally is not priced like a plain text completion.

Why it matters: use Flash-Lite for work that has a schema, a test, or a deterministic downstream check, not as a blanket replacement for the model that writes your production patch. Price the full request path—including searches—before declaring a workflow inexpensive. [Gemini 3 developer guide; Gemini API pricing]

Mistral Small 4 makes a credible cheap lane for structured coding chores

Mistral Small 4, released March 16, lists $0.15 per million input tokens and $0.60 per million output tokens, a 256,000-token context window, function calling, structured outputs, fill-in-the-middle completion, and batch support. Mistral describes it as a hybrid instruct, reasoning, and coding model with 119 billion total parameters but 6.5 billion active parameters.

At those rates, the 10-million-input, 2-million-output envelope is $2.70. That makes it plausible to run a lot of constrained attempts—generating table-driven test cases, normalizing configuration files, or proposing codemod transformations—but a low unit price does not cover the engineering cost of finding subtle semantic errors in a patch.

Why it matters: this is a useful candidate for a “produce structured candidate output, then verify it” stage in CI or an internal tool. Do not quietly promote it into an autonomous code-merging role just because the line-item cost is low. [Mistral Small 4 model card]

Hosted open-weight models are now a cost and latency option, not just an ideological choice

Groq’s production catalog lists OpenAI’s GPT-OSS 20B at $0.075 input / $0.30 output per million tokens and a published speed of roughly 1,000 tokens per second; GPT-OSS 120B is listed at $0.15 / $0.60 and about 500 tokens per second. The 20B model’s token envelope for 10 million input and 2 million output tokens is $1.35, which changes the economics of high-volume drafting, routing, and formatting.

The limitation is not subtle: a hosted open-weight model is not automatically competitive with a frontier model on difficult multi-file reasoning, unfamiliar frameworks, or an agent that must recover from bad tool output. Groq also distinguishes production and preview models, so pin a production model ID and treat a provider catalog change like any other dependency change.

Why it matters: test an open-weight hosted model where a wrong answer is cheap to reject, then measure task success, retry rate, and wall-clock time alongside token spend. The cheapest completion is expensive if it creates the second human review pass you were trying to eliminate. [Groq supported-models documentation]

The small-team play is routing and guardrails, not an all-in model swap

The durable operational change is to make “cheap first” safe. Put a budget and deadline on agent runs; pass stable repository instructions through prompt caching; use batch for noninteractive queues; and require tests, parsers, or human review before a low-cost model can modify an important branch. A shell command such as npm test -- --runInBand or git diff --check is often a better gate than asking a second model whether the first model did good work.

Why it matters: lower inference prices let a three-person team afford more attempts and better automation, but they do not remove the need to bound context, tool usage, or authority. The win is spending frontier-model money only where judgment is genuinely scarce—and having a measurable fallback when the cheaper route fails. [OpenAI API pricing; Anthropic pricing list; Gemini API pricing]