llama.cpp now asks each chat template which reasoning efforts it accepts, and the answers rarely match

"Reasoning effort" looks like a standard knob. OpenAI's API has it, most open reasoning models now claim to support it, and client UIs offer a tidy Low/Medium/High menu. llama.cpp's v0.8.0 release, tagged on October 28, includes a small server change that shows how untidy the knob is underneath: the server now works out, for each model's chat template, which effort levels actually do anything.

How the probe works

For open models, effort is not a sampler setting. It is text: the chat template takes the requested level and writes some instruction into the prompt, or doesn't. PR #31271 makes llama-server render a user turn for each candidate level (minimal, low, medium, high, xhigh, max) plus one made-up level. A level counts as honored if the template renders it without raising an error and the output differs from the made-up one, which shows what the template falls back to. Levels that produce identical prompts are treated as aliases, and the highest is kept. The result appears in /props as reasoning_effort_levels, next to the existing supports_reasoning_effort flag.

The levels the PR reports for a few templates are a decent illustration of why this was needed:

Seven templates, six different answers. A client that sends medium gets a real instruction from Qwen3.8 and only the template's fallback from DeepSeek, GLM, Mistral and Cohere2MoE.

What the old menu was doing

The companion UI change, PR #31273, explains the practical cost. llama.cpp's web UI had a Low/Medium/High/Max reasoning menu that always mapped to thinking_budget_tokens, a cap on how long the model may think, rather than to effort. On Qwen3.8, the PR says, picking Medium left the xhigh instructions in the prompt and then cut the think block off at 2,048 tokens. The user asked for a moderate amount of reasoning; the model was told to reason as hard as it could and then interrupted.

Now, when the server reports levels, the menu offers Default, Off and exactly those levels, and sends the choice as reasoning_effort with no token budget. Templates that report no levels, Gemma 4 among those tested in the PR, keep the old budget menu.

The probe tests templates, not models

The gpt-oss row is the one to read carefully. Its chat template writes "Reasoning: " + reasoning_effort into the system message verbatim, defaulting to medium. Any string renders, and every string renders differently, so the probe reports all six levels. The model card documents three: low, medium and high. Whether gpt-oss does anything sensible with Reasoning: max is a question about the model's training, and no amount of template rendering can answer it. The PR is upfront about this ("the value is inserted as is"), but a UI built on the list will still offer the extra options.

That is a reasonable trade for a heuristic that needs no per-model configuration, and it is a big improvement on a menu that silently meant something else. It also fits what a recent study found about the knob from the other side: turning effort up changes agent behavior in ways that are not a simple more-thinking-for-more-money dial (our coverage). If "high" means a different instruction in every template, and some templates accept levels their models were never trained on, comparisons of "the same effort setting" across open models deserve some suspicion.