How Production LLMs Reason Better At Inference Time
8 techniques, explained visually.
A smarter Claude model burns more tokens, not fewer
The above line sounds counterintuitive, but MCPMark V2 benchmarks confirmed this across 21 backend tasks.
And it’s not a minor 3-5% difference.
But 54% higher token usage.
The reason has nothing to do with the model itself.
Instead, it has to do with what the agent needs to know before it can start building.
When you’re building a full-stack app, CC must understand the entire backend, like:
what tables already exist
what RLS policies are active
what storage buckets are available
which auth providers are configured
and what edge functions are deployed
Most backends don’t hand over this info cleanly.
For instance, with Supabase, asking for OAuth setup via MCP returns the entire auth docs, including sections on email/password, magic links, phone auth, SAML, and SSO.
That’s 5-10x more tokens than the agent actually needed. And this happens on every MCP call across every domain.
The agent then discovers the state through separate calls to list_tables, execute_sql, and list_extensions, each returning a partial view.
Some info, like which auth providers are configured, isn’t queryable through MCP at all.
And when something breaks, Supabase returns the same error code whether the rejection came from the platform layer or from the function code.
The agent has no way to infer accurately, so it cycles through code-level fixes for a problem that might not be in the code at all.
A better model does not have a magical way to skip these gaps.
In fact, it tries even harder to fill them, which means more discovery queries, more reasoning, and more retries. That’s why the token cost went up with a better Claude model.
A smarter approach is actually implemented in InsForge, an open-source backend (self-hostable via Docker) that offers the same primitives as Supabase but structures everything around the assumption that an agent is operating the backend, not a human on a dashboard.
Before writing any code, a single CLI call returns the full backend topology in ~500 tokens.
The agent sees every table, auth provider, storage bucket, and available AI models in one structured response.
Instead of one broad skill like Supabase that triggers on everything, it has four narrowly scoped skills.
Creating tables only activates the CLI skill.
Debug skill only activates when code breaks.
Building frontend only activates the SDK skill.
Wiring third-party auth only activates the integrations skill.
This keeps the agent’s cognitive load lean since it only loads what matches the current task.
The CLI returns structured JSON with semantic exit codes on every operation, so the agent always knows whether something succeeded or failed and why. There are no ambiguous 401s that may indicate three different things.
We tested both backends on the same full-stack RAG app and recorded the full sessions.
Supabase:
consumed 10.4M tokens
needed 10 manual interventions
InsForge:
consumed 3.7M tokens
completed the entire build without any errors
This isn’t a Supabase-specific problem. Most backends were designed for humans who can see dashboards and interpret raw errors.
When an agent operates the backend instead, every missing piece of context needs a discovery call, and every ambiguous error enters a retry loop.
Fixing this requires giving agents structured backend context before they start writing code.
InsForge is an open-source implementation of exactly this, and you can self-host it via Docker.
GitHub repo (14k+ stars): https://github.com/InsForge/InsForge
(don’t forget to star it ⭐ )
You can read our walkthrough on building the full-stack RAG with Supabase and InsForge in this newsletter issue →
How production LLMs reason better at inference time
There are largely just two ways to make an LLM think harder at inference:
Parallel scaling samples the same prompt several times and selects one result.
Sequential scaling extends one trajectory further before emitting an answer.
Tree methods compose both, so they inherit both failure modes:
None of them alter the weights. It all runs in the prompt and in the orchestration around the call, which is why it is the first thing teams try when accuracy falls short.
But it is also the layer where the cost recurs.
Essentially, training cost is paid once and amortized across every future call, while inference-time compute is charged per request, forever.
The additional cost is often helpful though since several papers have found that under a FLOPs-matched comparison, test-time compute outperformed a 14x larger model on problems where the smaller model already had non-trivial success rates.
Let’s cover the techniques under each mechanism below:
1) Chain of thought
The model is prompted to think step by step, so it writes out intermediate steps before committing to an answer.
Reasoning models already do this on their own, so prompting for it separately mostly adds tokens.
2) Majority voting
Run the same prompt several times at nonzero temperature, then return whichever final answer appears most often.
It does not need any reward model since agreement directly acts as a signal.
Voting fixes random mistakes and does nothing about consistent ones. If the model misreads the problem the same way every time, all the samples agree, and the vote returns that answer with higher confidence.
It also needs answers that can be checked for equality, which rules out open-ended text.
3) Best-of-N
Generate N complete answers, score each one with a reward model, keep the highest scorer.
A paper measured that true reward rose, peaked, then declined as optimization pressure on the proxy increased. So past some N, the search starts finding answers the reward model likes rather than answers that are right.
The reward model also has to judge better than the model generates, which is the harder half of the setup.
4) Extended thinking
The model thinks for a set number of tokens before answering, with the caller setting the budget through budget_tokens, thinkingBudget, or reasoning_effort.
R1-Zero went from 15.6% to 71.0% on AIME 2024 through RL alone, with no external scaffolding at inference.
That said, more thinking is not always better. Anthropic’s inverse scaling work built tasks where accuracy drops as traces lengthen, including counting problems seeded with irrelevant numbers and constraint puzzles where the model reopens deductions it had already solved.
5) Self-refinement
The model writes an answer, critiques its own answer, then rewrites it, and the loop repeats.
However, this has the weakest evidence in this lift because in GSM8K, GPT-3.5 corrected 7.6% of its wrong answers and changed 8.8% of its correct ones, making the loop net negative.
6) Tree of Thought
While Majority voting varies the final answer, Tree of Thought varies the steps of reasoning at each point and then picks the best path overall.
At every reasoning step, the model explores multiple possible directions. These branches form a tree, and a separate process evaluates which path seems the most promising at a particular timestamp.
Think of it like a search algorithm over reasoning paths, where we try to find the most logical and coherent trail to the solution.
It’s more compute-intensive, but in most cases, it significantly outperforms basic CoT.
7) Beam search with a process reward model
Keep K partial solutions alive and score every step as it is written, instead of waiting for the final answer.
A paper found it ahead of best-of-N at small budgets and below it as budgets grow, with over-optimized runs producing repetitive steps or collapsing to one or two steps.
It works well on medium-difficulty problems, over-optimizes on easy ones, and no method works on the hardest tasks.
8) MCTS
Pick a promising partial path, extend it, run it out to a full answer, push the score back up the path, and repeat many times.
DeepSeek tried this for R1 and dropped it. The number of possible next tokens is far larger than the number of legal moves in a board game, and training a value model good enough to guide that search became a blocker.
They dropped PRMs for related reasons, since step correctness is hard to define, labeling does not scale, and the policy exploits the scorer.
DeepSeek solved this by placing the search into the weights with rule-based rewards instead.
To dive deeper, we covered all of this in depth across 13 parts of our RL series:
Part 1 covers the foundations, the agent-environment loop, and the reward hypothesis →
Part 5 covers function approximation and what breaks when states stop being enumerable →
Part 8 covers PPO and why it became the default for LLM post-training →
Part 9 covers RLHF, preference data, and reward model training →
Part 10 covers verifiable rewards and GRPO, which is the rule-based path R1 took →
Part 12 covers environments, trajectories, and the training loop →
Part 13 covers how AI teams use RL in production, including Cursor’s five-hour checkpoint loop →
Good day!