Claude Code’s model picker currently has six options sitting in it: qwen3.8-max, glm-5.2-fast-preview, glm-5.2, deepseek-v4-pro-0813, qwen3-coder-next, and grok-4.6. All of them are routed through my local CPA, a local model gateway, to their respective upstream providers. In day-to-day use, my impressions were vague: “this one feels faster,” “that one feels smarter.” Gut feel is unreliable, so I spent an evening putting them on the same starting line and benchmarked speed, thinking time, long-input handling, and a 9-question auto-graded mini IQ test, then cross-checked the results against public leaderboards.

First, a note on methodology and definitions, to avoid misreading the numbers:

• Speed test: direct connection to the local gateway, zero load, streaming responses. Timing was split into three parts: “thinking time” (the gap between the first content block and the first visible token), “time to first token” (from pressing Enter to seeing the first token), and “output rate” (pure generation speed during the streaming phase).
• IQ test: 9 zero-shot questions, auto-graded: 4 coding tasks (easy/medium/hard/bug fix, with real unit tests), 2 math questions, 1 logic question, 1 JSON-format-discipline question, and 1 “system instruction priority” question.
• Public benchmark scores are used only as a reference, not as the basis for my conclusions.

1. The truth about speed: waiting = thinking + fluency

The “model tok/s” numbers people often post only tell half the story. In the real experience, after you press Enter, you first sit through thinking time, and only then do you get token streaming speed. Once these two are measured separately, the personalities of the six models become obvious:

• qwen3-coder-next is lightning-fast: zero thinking time, first token in just 0.3 seconds, and 220 tok/s output. By the time you type ten characters, it has already finished.
• glm-5.2-fast-preview is the balanced one: 0.8 seconds of thinking, 1.8 seconds to first token, 114 tok/s output; for Chinese output, it is the fastest in the pool at 250 tok/s.
• qwen3.8-max is a heavy thinker: for English prompts, it thinks for 42.5 seconds before answering. Whether it is right or wrong is a separate issue—the wait alone is enough to discourage daily use. Chinese is somewhat better at 22.5 seconds.
• deepseek-v4-pro-0813 changes behavior by language: English requires 20 seconds of thinking, while Chinese is almost instant—1.6 seconds to first token and 91 tok/s.
• grok-4.6 buffers the whole response: it is not that it does not think; rather, through the account-pool route, it appears to accumulate the entire answer and then dump it all at once. Time to first token is 30–47 seconds, with a blank screen in between. It also ignores max_tokens—ask for 2048 and it gives 4721—so there is a risk of burning through quota.

2. Mini IQ test: only one perfect score, and two models fell into the “format lottery”

All 9 questions were zero-shot and auto-graded. Results below: ● passed, ○ failed.

Three points worth highlighting:

1. There was only one perfect score: qwen3.8-max spent 42 seconds thinking and earned the only 9/9. For it, thinking is not just decorative—it really does improve accuracy.
2. grok-4.6 coding is a lottery: it failed two coding questions due to Python indentation errors (IndentationError), while another coding question was completely correct. Looking at the raw output, the indentation had been stripped: def summarize(...):\n s = sum(nums). This is not a case of the model being unable to write the code; it is that, somewhere in this account-pool stack, the wrapper layer for some accounts strips leading spaces. With thousands of accounts taking turns handling requests, you never know whether the next one will be a “normal account” or an “indentation-stripping account.” The model itself is strong—its public score is tied with GPT-5.6 Sol Max—but on this route, using it for coding is a lottery.
3. The four models that scored 8 failed the same question, but not because they were dumb: on the “system instruction priority” question, all four chose to point out the instruction conflict and refused to blindly comply. That is a trace of safety training, not a capability failure.

One more note: qwen3-coder-next was the only model without thinking enabled. Its math results changed between two runs—the first time it missed the congruence question, the second time it missed the discount question. Models that do not think can fluctuate on individual questions, so do not convict them based on a single item.

3. Long context: others read a history in 1.8 seconds; grok takes 12.8 seconds

Claude Code conversations easily run into hundreds of thousands of tokens. Long-input processing speed is the hidden bottleneck that makes “larger context = sluggish interaction.” I tested with a 20K-token input:

The Bailian-backed glm-5.2-fast-preview was the fastest at 1.2 seconds, around 16K tokens/second. grok-4.6, via the account-pool route, took 12.8 seconds. That is an order-of-magnitude gap for the same job, and when choosing a model for million-token conversations, this row matters.

4. Summary: capability × responsiveness map

Plot the IQ score on the vertical axis and English time to first token on the horizontal axis, using a logarithmic scale, and you can immediately see where each model sits:

Cross-checking against public leaderboards: deepseek-v4-pro-0813 ranks first overall on LiveCodeBench with 93.5 and scores 80.6 on SWE-Bench Verified; qwen3.8-max is officially reported at 86.6 on Terminal-Bench and 93 on PaperBench; grok-4.6’s AA Intelligence Index score of 61 is tied with GPT-5.6 Sol Max. The public scores broadly match the ordering from my mini test. The only major reversal is grok: the model score is high, but the local route score is low. The bottleneck is not the brain; it is the pipe.

5. Selection checklist: which model to use for which job

For actual configuration, I would only change two places and leave the rest as-is: switch the default main model from qwen3.8-max to glm-5.2-fast-preview—the current setup effectively means “wait 43 seconds every time”; and switch the Haiku tier from grok-4.6 to qwen3-coder-next to avoid the indentation lottery.

6. A few bonus findings

• In the config, a name like qwen3.8-max[1M] with a [1M] suffix is actually Claude Code’s own client-side context-tier instruction. It is stripped before the request is sent—the gateway only recognizes the bare model name. If you are writing scripts that connect directly to the gateway, remember to use the bare name.
• kimi-k3 has already been removed from the pool; calling it returns a 400. The current OPUS tier has been switched to glm-5.2-fast-preview.
• There are still new models in the pool that I have not tested yet: the grok-4.20 family, including reasoning / multi-agent branches, gpt-5.6-luna/terra, and qwen3.8-2.4t-a95b.

Reproduction: all benchmark scripts and data are in ~/model-bench-2026-08-15/. Run python3 run_all.py and it takes about 15 minutes to produce a full round of results. The 9-question grader and speed-test logic are all open in the scripts. Feel free to pick apart my data.

Disclaimer: the IQ section is a 9-question mini benchmark, not a SWE-bench-level evaluation. The speed results are a snapshot from “the evening of 2026-08-15, on this route, under zero load”; upstream account pools and load will drift.