Reasoning models
Models that think before answering: chain of thought, spending compute at inference time, reinforcement learning, and how thinking is controlled and paid for.
Start with chain of thought
Even very large models struggled with multi-step problems like math word problems. Wei and colleagues showed that if you prompt a model to produce a chain of thought, a series of intermediate reasoning steps, performance on arithmetic, commonsense and symbolic tasks improves markedly1. With just eight exemplars, a 540B-parameter model reached state-of-the-art accuracy on the GSM8K math benchmark at the time of publication1. The reasoning ability emerged in sufficiently large models via prompting alone1.
The mechanism is mundane and powerful: each generated token is more computation. Writing out steps lets the model do work it cannot do in a single forward pass.
Spending compute at inference time
If thinking longer helps, how should you spend extra compute? Snell and colleagues studied scaling inference-time computation and found that the effectiveness of different approaches varies critically with prompt difficulty. That motivated a “compute-optimal” strategy that allocates test-time compute adaptively per prompt, improving efficiency by more than 4× over a best-of-N baseline2. In a compute-matched comparison, on problems where a smaller model already had non-trivial success, extra test-time compute could beat a model 14× larger2.
Diagram: my own, summarising the strategies compared in 2.
Training models to think
Prompting for chain of thought is optional; reasoning models are trained to do it. DeepSeek reported that reasoning can be incentivised with pure reinforcement learning, without human-labelled reasoning trajectories, and that behaviours emerged during RL training3. Their final R1 model reached performance comparable to OpenAI’s o1-1217 on reasoning tasks3. Training details are in How models are trained.
How thinking shows up in an API
Anthropic’s documentation describes thinking as the model working through a problem in its own words before answering: restating the question, trying approaches, checking intermediate results, and abandoning paths that fail4.
Diagram: my own, following the behaviour described in 4.
Controls have changed quickly, so check current docs:
- The older “extended thinking” form sets a thinking token budget with
budget_tokens, and Claude thinks against that budget before its final answer. Thinking tokens count towardmax_tokens, so the budget must leave room for the reply4. - That manual mode is now deprecated. Claude Opus 4.7 and later models reject it with a 400 error, and the documented replacement is adaptive thinking with an
effortsetting in place of a token budget4.
When to use a reasoning model
My working guide, not from the sources:
- Use it for multi-step maths, code with tricky logic, planning, and tasks where checking intermediate results matters.
- Skip it for simple lookup, formatting, classification, or latency-sensitive paths. Thinking costs tokens and time.
- Measure. Compare reasoning on and off on your eval set at several effort levels; do not assume more is better.
- In agents, reasoning helps with planning and recovering from errors, but each turn is slower. See ReAct and planning.
Reasoning does not remove failure modes. Next: Capabilities and failure modes.
Sources
- [1]Chain-of-Thought Prompting Elicits Reasoning in Large Language Models — Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou · arXiv, NeurIPS 2022 · 2022PARAPHRASED
- [2]Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters — Charlie Snell, Jaehoon Lee, Kelvin Xu, Aviral Kumar · arXiv · 2024PARAPHRASED
- [3]DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — DeepSeek-AI · arXiv · 2025PARAPHRASED
- [4]Extended thinking — Anthropic, Claude Platform docsPARAPHRASED