The Reward Signal Problem for Agents
The full RL nanodegree, covered with implementation.
The reward signal problem for agents
Part 11 of the Reinforcement Learning course is available now.
It tackles what is arguably the hardest open problem in agentic RL right now. GRPO needs a reward for every response. For math and code, a verifier supplies one. But what about the majority of real agent tasks where no answer key exists?
Read Part 11 of the RL course here →
It covers:
Why GRPO is agnostic to reward source
Verifiable vs. non-verifiable tasks
Why hand-written reward functions break
Constitutional AI and universal verifiers
LLM-as-a-judge as the core solution
Known judge biases and when not to use one
Hands-on LLM-as-judge scorer on a RAG task
Read Part 11 of the RL course here →
Why care?
The biggest bottleneck in training agents with RL right now is not the optimization algorithm. GRPO/PPO/DPO all work well.
Instead, the bottleneck is the reward signal.
For math and code, this was solved cleanly since a verifier can check correctness, and that becomes the reward.
But when you move to tasks like RAG, customer support, summarization, or any agent that produces free-form output, there’s no verifier.
And without a usable reward signal, GRPO has nothing to optimize against.
The current best answer is LLM-as-a-judge.
A capable model scores agent outputs relative to each other within each group, and GRPO normalizes those scores into advantages.
Research shows that strong judge models agree with human preferences at over 80%, which is roughly the rate at which two humans agree with each other.
This chapter covers the full landscape of covering hand-written functions and Constitutional AI to LLM judges, along with a hands-on implementation that builds a judge scorer from scratch and maps it into signals GRPO uses.
Over to you: What topics would you like us to cover in this RL series?
Good day!
Strands Agents: The open source agent harness SDK
Most SDKs give you tools to build an agent. Strands gives you tools to build an agent harness so you can control what it does, when it does it, and how far it goes.
Build an agent harness. Control it end-to-end.
Thanks to AWS for partnering today!
Build a 3Blue1Brown video generator using Strands
To learn about how to use Strands in practice, continue reading...
The challenge with most agent frameworks today is that they still force devs to hardcode workflows, decision trees, and failure paths.
That works until the agent sees something you did not anticipate.
As a solution, instead of over-engineering orchestration, newer agent systems lean into the reasoning capabilities of modern models and let them drive the workflow.
You define the goal, provide tools, and allow the agent to decide how to get there.
Strands Agents from AWS is a good example of this shift.
It is not about adding more rules or abstractions, but about letting the model plan, act, and recover dynamically.
Today, let’s do a hands-on walkthrough on this to learn how you can build Agents with a model-driven agent loop.
Everything will run 100% locally!
Let’s begin!
Implementation
Start by setting a uv environment and installing the required frameworks:
Make sure you have Ollama installed and its server is running locally.
Now building and running an Agent is as simple as these few lines of code:
You can print metrics, like token usage with result.totalTokens, run-time with result.latencyMs, or run result.get_summary() get a detailed nested JSON summary with tool calls, traces, etc.
While this is a minimal example, notice that there are no system prompts, no routing logic, no hardcoded steps, and no predefined workflow.
It simply defines three things: the model, the tools to use, and the task.
From there, the agent loop takes over.
At the time of Agent invocation, the model decides whether it needs a tool, which tool to use, how to structure the input, and when to stop.
Everything is model-driven.
So instead of encoding decision trees like “if math then calculator” or “if API fails then retry,” the model handles planning and execution dynamically. The workflow emerges at runtime based on the goal and available tools.
Strands Agents shifts agent development away from brittle orchestration and toward letting modern LLMs do what they are already good at: reasoning, planning, and adapting step by step.
Let’s look at a more concrete and real-world example.
We’ll build an MCP-powered Agent that can create 3Blue1Brown-style videos for us by just taking a simple prompt.
Yet again, the process stays similar:
In the above code:
An MCP client connects to an MCP server. This server has a tool that executes the Manim script generated by the model.
We extract the tool via
list_tools_sync()method and define the Agent with these tools.Finally, we execute the Agent!
This produces the following video output:
Notice that we defined the Agent with only tools. No system prompts or workflow rules. The model decides when and how to use Manim based purely on the task.
And a single natural language call that drives the entire flow. The model plans the steps, generates the Manim scene, and invokes the right tool autonomously.
Lastly, if needed, you can also deploy the Agent over the Amazon Bedrock AgentCore Runtime in a few lines of code!
It is a secure, serverless runtime purpose-built for deploying and scaling dynamic AI agents and tools using any open-source framework, including Strands Agents, LangChain, LangGraph, and CrewAI.
Here’s a simple example:
In the above code, we:
Import the Bedrock AgentCore runtime alongside Strands
Initialize a
BedrockAgentCoreAppthat handles scaling, security, and the request lifecycle.Define a Strands
Agentexactly the same way as before.Register a single entrypoint function that receives user input, forwards it to the agent, and returns the response.
This example above shows what makes Strands Agents genuinely interesting. You are not assembling an agent out of prompts, routers, and special cases.
You are defining capabilities and letting the model reason its way through the task, which lowers the barrier to building useful agents while still leaving plenty of room for control, customization, and experimentation.
A minimal setup can already produce surprisingly capable behavior, and as you add better tools or models, the agent naturally becomes more powerful without rewriting the workflow.
👉 Over to you: What would you like us to build next with Strands Agents?
Good day!