How do AI teams use RL in production?
The full RL nanodegree, covered with implementation.
How do AI teams use RL in production?
Part 13 of the Reinforcement Learning course is available now.
Here, we look at what actual AI labs and companies are actually doing with everything we learned across twelve chapters.
Read Part 13 of the RL course here →
It covers four case studies, each showing a different role RL plays in the industry:
Cursor’s real-time RL loop that ships improved checkpoints every five hours
How frontier labs turned verifiable rewards into reasoning and agentic capabilities
Scale AI’s enterprise results, where a 4B model beat GPT-5 on domain-specific tasks
The emerging market where RL environments are built, priced, and sold as products
Read Part 13 of the RL course here →
Why care?
Across all twelve chapters of this series, we built the full mechanics of RL piece by piece.
We covered bandits, value functions, Bellman equations, policy gradients, PPO, GRPO, judges, trajectories, and environments.
One real test of understanding RL is whether you can read a company’s blog post about their training pipeline and immediately identify the environment, the reward source, the trajectory structure, and the failure modes they are likely facing.
That is what this chapter exercises.
We cover stories of very different companies, each using RL in a way that maps directly onto concepts from this series.
The vocabulary you built is the same vocabulary these teams operate in, like reward hacking, credit assignment, environment design, and the tension between learned and verifiable rewards.
This is the final chapter, and this is what we have covered in the full course:
👉 Over to you: What topics would you like us to cover in this RL series?
LLM Routing vs Mixture of Agents
When you’re building a system that needs to handle diverse queries, two architectural patterns are considered.
One picks the best model for each task, and the other gets multiple models to collaborate on the same task.
Both involve multiple LLMs, but what they do with those models is completely different.
The visual below explains how each one works:
In LLM routing, an intent classifier reads the prompt, and a cost ranker weighs latency, token price, and capability, so only one model gets the call while everything else stays idle.
The prompt is classified by domain and action.
A cost ranker picks the best model given latency and price constraints.
One model responds, and the rest never see the query.
In Mixture of Agents, a set of consulting models each analyze the query independently and hands their output to a single aggregator, which uses every perspective to write the final answer and make the tool calls.
The consulting models never see each other’s analysis, only the aggregator does.
On HermesBench, Nous Research reports a MoA preset scored higher than either model alone by roughly six points and eight to eleven percent, and no individual model in that setup reached the same result on its own.
LLM routing and MoA are not competing approaches but rather two different tools for two different problems.
Routing is about picking the right model for the task, while MoA is about combining multiple models to exceed what any one can do alone.
We wrote a full deep dive on LLM routing with Plano, including a working demo, in a recent article.
Here’s the Plano GitHub repo: https://github.com/katanemo/plano (Don’t forget to star it ⭐️)
And here’s the article we wrote →
P.S. For those wanting to develop “Industry ML” expertise:
At the end of the day, all businesses care about impact. That’s it!
Can you reduce costs?
Drive revenue?
Can you scale ML models?
Predict trends before they happen?
We have discussed several other topics (with implementations) that align with such topics.
Here are some of them:
Learn everything about MCPs in this crash course with 9 parts →
Learn how to build Agentic systems in a crash course with 14 parts.
Learn how to build real-world RAG apps and evaluate and scale them in this crash course.
Learn sophisticated graph architectures and how to train them on graph data.
So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.
Learn how to run large models on small devices using Quantization techniques.
Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using Conformal Predictions.
Learn how to identify causal relationships and answer business questions using causal inference in this crash course.
Learn how to scale and implement ML model training in this practical guide.
Learn techniques to reliably test new models in production.
Learn how to build privacy-first ML systems using Federated Learning.
Learn 6 techniques with implementation to compress ML models.
All these resources will help you cultivate key skills that businesses and companies care about the most.