Open-Weight Coding Models Are Closing the Gap, but Not Evenly Across Benchmarks

Open-weight coding models have stopped being merely the cheap fallback for autocomplete and small patches. Kimi K3, GLM-4.7, and Qwen3-Coder-Next all target the agent loop directly: inspect a repository, call tools, edit files, run commands, and keep going. But their reported gains land on different tests, with different agent harnesses and very different deployment requirements, so “near frontier” is useful only after you name the task.

Kimi K3 brings open weights into the long-horizon agent comparison

Moonshot’s Kimi K3 is a 2.8-trillion-parameter mixture-of-experts model with 104 billion active parameters, a one-million-token context window, and native image input. Its published coding results are close to the leading closed models on several agent-oriented tests: 67.5 on DeepSWE, 88.3 on Terminal-Bench 2.1, and 81.2 on FrontierSWE, while trailing on others such as PostTrainBench. Those are not local-model numbers: the weights use Moonshot’s own Kimi K3 License rather than a standard permissive license, and a 2.8T model is an infrastructure decision before it is an editor setting.

Why it matters: K3 is evidence that open weights can be credible for long terminal sessions, but it is a candidate for a hosted or serious self-hosted service—not something to treat as a drop-in laptop replacement for a coding subscription.

GLM-4.7 makes the strongest case for tool-aware open deployment

Z.ai’s GLM-4.7 reports 73.8% on SWE-bench, 66.7% on SWE-bench Multilingual, and 41% on Terminal-Bench 2.0, with explicit support for thinking between tool calls and retaining reasoning across turns. The repository ships a concrete vLLM configuration—--tool-call-parser glm47, --reasoning-parser glm45, and --enable-auto-tool-choice—which is more useful than a vague claim of agent compatibility. The catch is deployment size: its own requirements list four H100s for the FP8 355B-A32B model under its baseline setup, while the smaller GLM-4.7-Flash is the one-model-on-one-H100 option.

Why it matters: if your team owns GPU serving and wants parser-level control over a terminal agent, GLM-4.7 is worth evaluating; if you do not, the headline SWE-bench number is not a reason to inherit an H100 fleet.

Qwen3-Coder-Next targets the local and swappable-agent tier

Qwen3-Coder-Next is the more practical counterweight to the giant releases: it is built on Qwen3-Next-80B-A3B-Base, exposes a 256K-token context window, and has FP8 and GGUF distributions alongside the full checkpoint. Qwen says it is designed for coding agents and local development, and documents compatibility with Qwen Code, Cline, and Claude Code-style workflows. Integration is not entirely generic, though: function calling depends on Qwen’s new tool parser in SGLang or vLLM, it requires the new tokenizer, and the model operates only in non-thinking mode.

Why it matters: this is the open-weight option to trial when model portability and local inference matter, but run its actual tool-call format through your agent before replacing a model that already works with your harness.

The benchmark gap is now mostly a harness gap

The most consequential detail in the new scorecards is not the top-line percentage but the agent that produced it. Moonshot’s K3 documentation explicitly says its DeepSWE result uses Kimi Code, compares other models through mini-SWE-agent or their published results, and uses Kimi Code, Claude Code, or Codex across other coding evaluations; it even reports a 67.5 DeepSWE result in one setup and 67.3 with mini-SWE-agent. That disclosure is better than pretending the harness is neutral, but it means a K3-versus-GLM-versus-closed-model ranking is still a bundle of model, tool permissions, context management, prompting, retries, and test-time compute.

Why it matters: use public benchmarks to choose a short list, then run the same task corpus, container image, timeout, tool policy, and agent loop you expect to use in CI—otherwise you are selecting an integration, not just a model.