We spent a morning with Claude Fable 5, and it left us thinking about one thing, i.e., what an agent is allowed to do while it runs on its own.

Fable 5 runs for hours, holds a goal for days, and takes action without checking in. So the question stopped being what an agent can build. It became what it can touch while it runs.

We tested this with a plain internal tool.

Fable 5 wrote the SQL, pulled real records, and added a button that issued a refund straight to the database, all in one pass.

The build was easy, but deciding who can open it, who can run that refund, and keeping a record of what happened was the part the model could not handle on its own.

That part lives in the runtime, not the model.

We deployed the same agent into Retool, and it picked up company SSO, a role check on the refund, and an audit log of every query and action. The app didn’t change, but where it ran did.

Retool calls this building inside the lines, and you’ll notice the difference once you watch the same agent run both ways.

You can try it out yourself here →

Thanks to Retool for partnering today!

AI concepts every AI engineer should know

Two teams can build on the same base model and ship completely different products.

The model is a fixed input. What differs is the eight layers of engineering wrapped around it, from how tokens get served to how the agent loop is controlled.

We mapped the full set into a single diagram:

Model foundations cover how a model turns text into probabilities:

• Tokenization splits text into subword unit before the model ever sees it, and token count drives both cost and context limits.

• Embeddings map those tokens to high-dimensional vectors where similar meanings sit close together.

• Pretraining learns language from raw text while post-training (SFT, RLHF) shapes behavior and alignment.

• The context window is the fixed token budget the model attends to at once, shared across prompt, history, and output.

• Logits are the raw scores over the vocabulary that sampling (explained below) turns into the actual output token.

• Sampling controls how the next token is drawn from the distribution, with temperature and top-p trading determinism for diversity.

We covered this layer in depth (with code) here →

Inference and serving cover the stack that turns weights into cheap/fast tokens:

• Prefill processes the whole prompt in parallel and is compute-bound, while decode generates one token at a time and is memory-bound.

• The KV cache stores attention keys and values for past tokens so they aren’t recomputed at every step.

• Prompt and prefix caching reuses the KV state of a shared prefix, which makes fixed system prompts effectively free after the first call.

• Speculative decoding uses a small draft model to propose several tokens that the main model verifies in parallel for a net speedup.

• Continuous batching fills GPU slots with new requests the moment others finish, instead of waiting for a whole batch to complete.

• Quantization stores weights in fewer bits (FP8, AWQ) to cut memory and speed up compute, with FP8 running natively on recent GPUs.

• Paged Attention applies OS-style paging to the KV cache to remove fragmentation, and it’s the core trick behind vLLM.

• TTFT and TPOT measure prefill and decode speed respectively, and both trade against raw throughput.

We covered this layer in depth (with code) here →

Context engineering involves managing what the model sees at the moment it acts:

• Context budgeting treats the window as a finite resource and spends tokens only on what raises answer quality.

• Context rot is the way output quality degrades as the window fills, often well before the hard limit.

• Lost in the middle describes how models attend most to the start and end of the context, so details buried in the middle get underweighted.

• Compaction and summarization condense old history into a high-fidelity summary so the agent can continue in a fresh window.

• Context offloading pushes large details to external files or stores and keeps only a reference in the window.

• Just-in-time retrieval loads data at the step that needs it rather than front-loading everything upfront.

• Structured note-taking lets the agent write persistent notes outside the window and read them back as needed.

We covered this layer in depth (with code) here →

Agents and harness engineering a stateless model into something that finishes tasks.

• The agent loop runs the think, act, observe cycle (ReAct or TAO) until the task is done.

• Tool use and function calling let the model emit structured calls that the harness executes, feeding the results back to the model.

• A thin harness trusts the model and keeps infrastructure minimal, while a thick harness encodes control in code and leaves less to the model.

• Subagents and orchestration spin off focused agents with their own context so the main agent stays lean.

• MCP is a standard interface connecting models to tools and data, replacing N×M custom integrations.

• Skills, hooks, and state add reusable capabilities, lifecycle triggers, and persisted memory that survive across steps.

• Planning versus reacting is the choice between committing to a plan upfront and deciding each step live, with different speed and cost.

• Verification loops use rules, tests, or an LLM judge to gate the agent’s output before it counts as done.

We covered the Anatomy of Agent Harness here →

Retrieval and memory feed the model facts it was never trained on.

• RAG pipelines retrieve relevant chunks at query time and add them to the prompt before generation.

• Chunking and re-ranking split documents into units and then reorder retrieved candidates by true relevance.

• A vector DB stores embeddings and serves nearest-neighbor search at scale.

• Hybrid search combines keyword and vector matching to catch both exact terms and semantic matches.

• Knowledge graphs model entities and relationships so retrieval can follow connections, not just similarity.

• Episodic and temporal memory keep what happened and when, so the agent recalls past sessions and how facts changed.

• Agentic retrieval lets the agent issue and refine its own queries instead of running a single fixed lookup.

We covered this layer in depth (with code) here →

Adaptation and training change the weights when prompting and context aren’t enough:

• Fine-tuning (SFT) trains the model on input-output examples to teach format and behavior.

• LoRA and QLoRA (PEFT) train small adapter weights instead of the full model, which cuts cost and memory sharply.

• RLHF and DPO optimize against human preferences, with DPO skipping the separate reward model.

• GRPO and reward models train against a reward signal, which works when you have a scoring function but no preference labels.

• Distillation trains a smaller student to match a larger teacher, keeping most of the accuracy at lower cost.

• Synthetic data generates training examples with a model when real labeled data is scarce.

We covered this layer in depth (with code) here →

Evaluation and observability help track whether a change helped or quietly broke something.

• Offline and online evals run fixed test sets before shipping, and live metrics from real traffic after.

• LLM-as-judge uses a model to score open-ended output that rules can’t grade.

• Agent trajectory eval judges the full path an agent took, not just the final answer.

• Tracing and spans record each step, tool call, and token so you can see where a run went wrong.

• Token and cost tracking attributes spend per request and per step to find what’s expensive.

• Regression testing re-runs a benchmark after a prompt or model change to catch silent breakage.

We covered this layer in depth (with code) here →

Safety, security and reliability keep the system honest in front of real users.

• Prompt injection is untrusted input that smuggles in instructions to hijack the model’s behavior.

• Jailbreaks are crafted prompts that get the model to bypass its own safety constraints.

• Hallucination mitigation uses grounding, retrieval, and verification to catch confident wrong answers.

• Structured outputs constrain generation to valid JSON or a schema so downstream code can parse it.

• PII and data privacy controls detect and redact sensitive data before it enters or leaves the model.

• Fallbacks and rate limits add multi-provider failover and request caps that keep the system up under load.

• Guardrails are input and output filters that block unsafe or off-policy requests and responses.

We covered this layer in depth (with code) here →

The inference layer is where most production cost hides.

On an H100 running Llama 70B, a single request hits 92% GPU utilization during prefill, then drops to 28% during decode on the same hardware a moment later. The workload changed, not the GPU.

No single technique moves that number much. Stacking eight or nine of them, across compression, attention, decoding, caching, and routing, is what closes a 5-8x cost gap against naive FP16 inference.

We broke down all 72 of these techniques, with the engineering logic behind each, in this issue and across the full LLMOps crash course.

For routing trivial queries to cheaper models, the advisor strategy is another interesting path to start with.

👉 Over to you: which of the eight layers do you spend the most time in?

Thanks for reading!

They tracked 807 GitHub repos that adopted Cursor.

Each was matched against a similar repo that did not, so the difference reflects the agent, not the project.

In the first month after adoption, the agent-assisted repos added 3 to 5x more code. That gain was real but short-lived, and it faded within two months.

What did not fade was the hit to code quality.

Static analysis warnings rose around 30% and code complexity around 41%, and both stayed elevated for the rest of the study.

The obvious objection is that this is a skill problem, fixable with sharper prompts and review.

But even after controlling for how much code was added, complexity still climbed faster in the agent-assisted repos than in the controls. The agent adds to the problem, not just the developer using it.

The instinct is to have the agent review the code it just wrote, or add a second agent to check it.

That does not work as well as expected because the model that wrote the code carries the same blind spots that produced the defects.

A second model trained on similar data shares most of them, so an AI reviewer misses what the AI author missed.

Catching this reliably needs deterministic code analysis, which applies fixed rules instead of another probabilistic guess.

SonarQube actually implements this as a plugin inside Claude Code.

It runs deterministic verification in the session, flagging issues, complexity, and exposed secrets after every edit across the project.

For instance, in the video below, I typed a prompt with a GitHub token in it, and SonarQube blocked it before it reached Claude. The token never entered the context window.

To get started in Claude Code, run /plugin, find sonarqube, install it, then run /sonarqube:integrate.

You can find the Claude plugin here →

And here’s the CMU’s study →

Thanks to Sonar for partnering with us!

Training an LLM to generate reliable structured output

An LLM has no inherent concept of a parser. It predicts the next token from the ones before it, and well-formed JSON is only one of the continuations that fit this format.

So a number written as a string fits just as well, and so does an object with a line of explanation in front of it.

More examples and a tighter prompt do push the failure rate down, but then it stalls.

DeepSeek-R1 showed a way around this. Training a strong model used to mean annotation pipelines, preference pairs, and a team of labelers.

DeepSeek replaced all of it with one function that checks whether an answer is right. If you can define correctness via code, you do not need the rest.

That is the idea behind GRPO. Instead of learning from examples, the model learns from a reward function you write.

For each prompt, it generates a few candidate answers. The reward function scores them, and the model is pushed toward the ones that score higher.

Today, let’s use this idea to fine-tune Qwen3-8B for JSON extraction. The loop runs from a local notebook while the model trains on remote H200s.

The reward function just checks whether each output parses and matches the schema.

Schema-valid output went from 62% on the base model to 82% after training, past GPT-4.1 on the same eval at 58%.

Before we dive into code, let’s see why the obvious approach stalls.

Problems with SFT

SFT learns by copying examples. If you show it correct completions, it gets good at producing output that looks like them.

Notice the word “looks.”

This implies that looking like valid JSON and being valid JSON are different goals, and SFT only optimizes the first one.

The loss is measured token by token, and a completion with one field typed wrong scores almost the same as a perfect one.

If you add more examples, the metrics do tick up, but then they flatten, because the limit is the objective, not the data.

Once you see the problem this way, the fix becomes immediately clear. You can define what “correct” means in code, and train the model against that definition directly.

This is what GRPO does.

It swaps labeled examples for a reward function. For each prompt, the model generates a small group of answers (usually 4-8). And then the reward function scores every one of them.

The scores are normalized inside the group. The model update then reinforces the answers that scored above the group’s average.

So the model is always comparing its own outputs against each other. It learns what “more correct” means for your task, not what “more similar to an example” means.

Here is how the reward function scores three different outputs for the same prompt:

• Output that doesn’t parse as JSON scores 0.0.

• Output that parses but fails the schema scores 0.5.

• Output that parses and matches the schema scores 1.0.

That middle score is important because without it, valid JSON with the wrong field types would score zero, the same as complete garbage.

The model would lose an important signal that a valid structure is already progressing. The 0.5 is what gives training something to climb toward.

Infra for GRPO

GRPO is heavier than SFT. On an 8B model, it needs H200s and runs for hours.

Every step, it generates several completions per prompt, scores all of them, and updates the weights. That repeats across the whole dataset, many times over.

So this is not something you can run on your laptop.

There is also a timing problem that SFT never has.

During rollout, the model samples answers from its current weights. During training, those same weights are changing underneath it.

If the inference side and the trainer fall out of sync, you sample from a stale model and train on answers that the current one would never give. This is where most custom RL setups fall apart.

Fireworks’ Training API handles both sides. You can write the training logic in Python on your own machine, and the infrastructure handles the rest.

It provisions the GPUs, runs the forward and backward passes, saves checkpoints, and resyncs the inference deployment after every step.

The setup is three steps:

• Write the reward function

• Upload the dataset

• And configure the run.

Let’s go through each one.

The training loop

Fireworks documents the full setup in their Training API docs. That includes rl_loop, the recipe that runs the entire GRPO loop.

Clone the cookbook and install the training dependencies:

git clone https://github.com/fw-ai/cookbook.git

cd cookbook/training && pip install -e ".[training]"

export FIREWORKS_API_KEY="your-training-scoped-key"

Step 1: Write the Reward Function

This is the only place your task is defined. The SCHEMA says what a correct output looks like, and score() checks each completion against it.

For invoice extraction, we pull four fields from raw text: vendor, date, amount, and currency.

import json
from jsonschema import validate, ValidationError

SCHEMA = {
    "type": "object",
    "required": ["vendor", "date", "amount", "currency"],
    "properties": {
        "vendor":   {"type": "string"},
        "date":     {"type": "string"},
        "amount":   {"type": "number"},
        "currency": {"type": "string"},
    },
    "additionalProperties": False
}

def score(completion: str) -> float:
    try:
        parsed = json.loads(completion.strip())
    except (json.JSONDecodeError, ValueError):
        return 0.0
    try:
        validate(instance=parsed, schema=SCHEMA)
        return 1.0
    except ValidationError:
        return 0.5

jsonschema handles the type checks, the required fields, and any nested rules in a single call.

For a different task, like SQL or tool-call formatting, you can swap in a new schema. The score() function stays the same.

Step 2: Prepare the Dataset

GRPO does not need labeled outputs. The dataset is just the prompts you would send in production.

The model writes its own completions during training, and score() grades them as they come.

We used 200 training prompts. They cover different vendor names, date formats, amount styles, and currency codes. We also set aside 50 eval prompts.

Variety matters more than volume here. Prompts that all look alike produce a model that breaks on real invoice variation.

{
  "messages": [
    {
      "role": "user",
      "content": "Extract the following fields from this invoice:
                  Bill from Acme Corp, dated 2024-03-15, total $1,250.00 USD.
                  Return valid JSON only."
    }
  ]
}
{
  "messages": [
    {
      "role": "user",
      "content": "Extract the following fields from this invoice:
                  Received from TechSupplies Inc on January 8 2024,
                  amount due: 340 euros. Return valid JSON only."
    }
  ]
}

Upload the dataset and wait for the READY state before the training job can use it.

from fireworks import Fireworks

client = Fireworks()
client.datasets.create(dataset_id="invoice-extraction-grpo-v1",
                       dataset={"exampleCount": "200"})
client.datasets.upload(dataset_id="invoice-extraction-grpo-v1",
                       file="./train_prompts.jsonl")

# poll client.datasets.get(...).state until "READY" before proceeding

Step 3: Configure and Run the Loop

rl_loop runs the whole thing. It provisions the trainer, schedules the rollouts, syncs the weights, and cleans up when the run is done.

You connect your score() function by wrapping it and assigning it to rl_loop.reward_fn. The wrapper gets both the completion and the dataset row, so you can reach ground-truth metadata if your reward needs it.

from training.recipes.rl_loop import Config, main
from training.utils import DeployConfig, InfraConfig, WeightSyncConfig
import training.recipes.rl_loop as rl_loop

# Wire your reward function to the training loop
def invoice_reward(completion: str, row: dict) -> float:
    return score(completion)

rl_loop.reward_fn = invoice_reward

cfg = Config(
    base_model="accounts/fireworks/models/qwen3-8b",
    dataset="invoice-extraction-grpo-v1",
    max_rows=200,
    epochs=1,
    completions_per_prompt=4,
    max_completion_tokens=256,
    temperature=1.0,
    max_seq_len=4096,
    policy_loss="grpo",
    output_model_id="invoice-extractor-v1",
    infra=InfraConfig(
        training_shape_id="accounts/fireworks/trainingShapes/qwen3-8b-128k",
    ),
    deployment=DeployConfig(
        deployment_id="invoice-extractor-v1",
        tokenizer_model="Qwen/Qwen3-8B",
    ),
    weight_sync=WeightSyncConfig(weight_sync_interval=1),
)

main(cfg)

A few of these settings to note here:

• dataset points to the dataset_id uploaded in Step 2. Fireworks pulls it from their storage directly.

• completions_per_prompt=4 sets the group size for GRPO. Production runs often use 8 to 16, which gives more signal per step at a higher compute cost. Four is enough here. The reward is clear-cut enough that even a small group shows real variance between answers.

• weight_sync_interval=1 resyncs the inference deployment after every step. That keeps rollout sampling from the exact model being trained. Production runs often set this to 4 or 8 for speed. For a short 200-step run, 1 gives the tightest feedback loop, which is what you want.

• One Qwen3 quirk to handle here is that it defaults to thinking mode and adds <think>...</think> blocks before the answer. Strip them at eval time with content.split(”</think>”)[-1].strip(). Suppress them in training by adding /no-think to the system prompt. Otherwise, the reward function reads the reasoning text instead of the JSON, and scores everything 0.0.

Results on the held-out eval

Base Qwen3-8B scores 62% schema-valid on the 50 held-out prompts. After GRPO training on Fireworks H200s, the fine-tuned model hits 82%.

That is past GPT-4.1 on the same eval, which lands at 58%.

Here is the baseline run first, on the 50 prompts the model never trained on.

And here is the same eval after training.

The trained model runs on a Fireworks serverless endpoint, at a fraction of GPT-4.1’s per-token cost. Latency is lower too, since the output is short and predictable.

The real difference shows up on messy inputs. A prompted general-purpose model starts to slip, while the trained model holds, because it learned the constraint instead of the examples.

This pattern works for any task where you can check correctness in code.

SQL that has to parse, API responses that must match a shape, tool calls, and code that has to pass a linter.

If you can score an output, you can train a model to chase that score.

What DeepSeek-R1 proved at the frontier scale holds for your own small task. The model you get has practiced your definition of correct, not memorized examples of it.

You can find the training API docs in this GitHub repo →

The full code is in the repo below. That includes the reward function, the training config, the dataset builder, and the eval script.

And you can find the finetuning code in our repo here →

Thanks for reading!

Your first agent: running in a few lines of code.

Your production agent: the same lines of code, scaled.

Strands grows with your use case, from a local prototype to a deployed system handling real workloads, without a rewrite in between.

Open source. Model-driven. Any backend.

Start building →

Thanks to AWS for partnering today!

Loop engineering: Design the system that prompts agents

In a hand-run agent session, you hold two jobs:

• deciding what the agent runs next

• and checking its output before the next step.

Loop engineering moves both into the system.

Essentially, a schedule decides what runs, a separate checker agent grades the output, and a file on disk holds the state that they both read.

Boris Cherny, who leads Claude Code, stopped prompting Claude by hand and now writes the loops that prompt it instead.

So today, let’s break down what a loop is built from, the split that does the real work, and what stays manual.

The six parts a loop are built from

A loop is assembled from five components plus a memory, and Claude Code and Codex both ship all of them now.

• Automations fire on a schedule and do discovery and triage on their own.

• Worktrees give each agent its own checkout, so two running in parallel don’t overwrite each other’s files. Leaving orphaned worktrees behind is the main way this turns messy.

• Skills hold the project knowledge the agent would otherwise re-derive from zero every run, like conventions, build commands, and review standards. An automation can fire a skill by name instead of carrying a wall of instructions that nobody maintains.

• Connectors wire the agent into your real tools over MCP, the issue tracker, the database, Slack.

• Sub-agents split the work, so one agent makes the change and a different one checks it.

• Memory is the sixth piece, giving you a dedicated knowledge graph (using Zep Graphiti, for instance) that lives outside the conversation and records what’s done and what’s next.

Here’s what you need to make sure when building these systems:

The maker-checker split

The split between the maker and the checker makes a loop safe to leave alone.

This is important because a model grading its own output is too lenient to catch its own bugs, so the checker runs as a separate agent, ideally a different model, scored against independent signals.

Its findings go back to the maker as the next instruction, and the cycle repeats until they come back clean.

Set the exit before you start

A loop with no stop condition churns and burns tokens, and the token cost climbs fast once sub-agents and long runs stack up.

One thing you can do in practice goes something like…fix major issues only, run one final pass, then stop after two loops, with “all tests pass and lint clean” as the explicit exit.

Decide this before the loop runs, not while it’s running.

State lives on disk, not in context

The model forgets everything between runs, so the state has to sit outside the conversation.

A markdown file or a knowledge graph can hold what’s done and what’s still open, and each run reads it first and writes back to it last.

The agent forgets between runs, but this does not, which is precisely what lets a loop pick up across days instead of resetting every session.

Loops are important on the boring, high-frequency checks you skip by eye, like a stale version string or a missing test, but not on work that needs judgment.

An unattended loop fails in two ways.

It reports done when nothing actually verified the work, which is what the separate checker is there to prevent.

And it merges code faster than you read it, so over weeks, you stop understanding your own codebase while every check stays green.

Green tests tell you the code passed the tests, not that you know what shipped, so the habit that keeps a loop honest is reading what it merges, not just watching the checks pass.

We’ll make this much more concrete by sharing a hands-on demo soon.

In the meantime, just remember that loop engineering is a simple trade where you stop being the one who prompts and become the one who designs the loop that prompts it.

👉 Over to you: where would you actually let an agent loop run unattended, and where would you keep your hand on every turn?

Thanks for reading!

P.S. For those wanting to develop “Industry ML” expertise:

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?

• Drive revenue?

• Can you scale ML models?

• Predict trends before they happen?

We have discussed several other topics (with implementations) that align with such topics.

Develop "Industry ML" Skills

Here are some of them:

• Learn everything about MCPs in this ​crash course with 9 parts →​

• Learn how to build Agentic systems in a crash course with 14 parts.

• Learn how to build real-world RAG apps and evaluate and scale them in this crash course.

• Learn sophisticated graph architectures and how to train them on graph data.

• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.

• Learn how to run large models on small devices using ​Quantization techniques.

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using ​Conformal Predictions.

• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.

• Learn how to scale and implement ML model training in this practical guide.

• Learn techniques to reliably test new models in production.

• Learn how to build privacy-first ML systems using ​Federated Learning.

• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

The voice agent stack is one of the few areas in AI where the demand is enormous, and the worked examples barely exist.

Speechmatics Academy has open-sourced a collection of runnable examples across batch, real-time, voice, and TTS, each standalone enough to clone a single folder and have a working pipeline in minutes.

Speechmatics Academy GitHub Repo

The integrations have complete loops with LiveKit, Pipecat, Twilio, and VAPI, covering WebRTC capture, turn detection, speaker focus, interruption handling, and function calling.

The use cases it teaches cover production territory, including SRT captioning, call-center topic detection, and HIPAA-friendly medical microbatching with Silero VAD chunking.

While everyone debates voice as the next interface, this is the repo that shows how to actually build for it.

You can find the GitHub repo here → (don’t forget to star it ⭐️)

Thanks to Speechmatics for partnering today!

Your agent harness should repair itself

When an AI agent fails in production, the observability tool only shows exactly what it did and almost nothing about how to fix it.

You get a clean trace of every model and tool call, the latency per step, and the token usage. What you don't get is why the trace broke, the change that fixes it, or any guarantee that the same failure won't recur.

So you scroll the trace span by span, form a theory, write a patch, and hope it doesn’t break something that already worked.

Then a new model ships with its own failure modes, and you run the loop again from the top.

Cursor recently shared how much engineering goes into the harness around their agent, the prompt layers, tools, and checks wrapped around the raw model. Every model upgrade and every new tool widens that surface.

Today, let’s look at how you can use Opik (open-source) to solve this.

The four-layer stack in Opik

For context, Opik is an open-source logging, debugging, and optimization platform for AI agents and LLM apps.

Opik is built around the premise that the observability loop should be automated, not staffed, which is why Opik’s architecture is one connected workflow.

Trace → Ollie diagnoses → Ollie proposes a fix → fix is applied and verified → Test Suite locks the failure as a regression test → back to Trace

Let’s look at each layer.

Layer 1: Tracing

Every LLM call, tool invocation, and retrieval step is instrumented automatically with a single decorator.

import opik

@opik.track
def my_agent(query: str):
    # your agent logic here
    ...

This works with LangGraph, CrewAI, and 50+ frameworks. Every trace records which agent configuration was active for full reproducibility when you need to rerun a failing input later.

Layer 2: Ollie

Opik’s Ollie is a recent addition that accepts a trace and fixes code. It is a coding agent built into Opik with full context.

If you run it on your project root, Ollie will:

• Read your source files

• Identify the exact lines responsible

• Propose a diff (nothing changes without your explicit approval)

Once you approve, Ollie reruns the agent against the exact inputs from the original failing trace, streams the new trace for side-by-side comparison, and locks the original failure as a regression case in your test suite.

Bad trace → root cause → diff → approve → rerun → regression locked

Without any code access, Ollie reads span trees, identifies failure modes, and explains the causal chain across every LLM call. If you ask it, “why did the final answer ignore the retrieved context?”, it walks the full span tree and surfaces the root cause.

Layer 3: Test suites

Eval workflows typically build a labeled dataset, define a numerical metric, and compare floats, but it doesn’t match how engineers think about quality.

You can replace it with plain-English assertions in Opik:

suite = opik.TestSuite("crm-agent-v2")

suite.add_assertion("The response must include specific deal details, not just a count")

suite.add_assertion("The response must never reveal unauthorized information")

suite.run_tests()

Opik converts those into LLM-as-a-judge checks under the hood and gives clean pass/fail per test case.

Here, every failing trace you debug automatically becomes a new test case, so the test suite naturally grows from real production failures.

Every cycle, the harness gets harder to break.

But even with a growing test suite, you still need a safe place to test changes before they ship. That’s what Layer 4 is for.

Layer 4: Agent sandbox

Most playgrounds are prompt playgrounds. Changing a system prompt and rerunning the LLM call answers the wrong question.

The production question is, what happens to the entire agent graph when I change something?

Opik’s Agent Sandbox runs the fully instrumented agent end-to-end inside the UI. Change a prompt, swap a model, add a tool, and watch how the whole system responds across the full spanning tree.

Every sandbox run produces a complete Opik trace.

In practice

The layers aren’t independent features. They’re one loop that you can instrument with @opik.track.

If something fails in production.

• Ollie reads the trace + source code and proposes a fix.

• You approve.

• Ollie reruns the agent in the Sandbox against the original failing input.

• Once fixes pass, it saves it as a new blueprint.

• The environment pointer promotes to staging.

• Finally, the original failure is locked as a regression test.

The next failure (if any) enters the same loop.

And in every cycle, the harness gets harder to break.

The whole stack we discussed today is open-source (19k+ stars), covering Tracing, Ollie, Test Suites, the Agent Sandbox, a 6-algorithm Agent Optimizer, and 50+ framework integrations.

You can self-host everything in three commands:

git clone https://github.com/comet-ml/opik
cd opik
./opik.sh

Here’s the Opik GitHub Repo → (don’t forget to star 🌟)

👉 Over to you: What’s the current state of observability in your agent stack, and where does the debugging loop break for your team right now?

Good day!

Every method up to this point learned a value function first and derived behavior from it. In this chapter, instead of scoring actions and picking the best one, we learn the choosing itself.

Reinforcement Learning Nanodegree Part 7

You can read Part 7 of the course here →

It covers:

• Why learning the policy directly matters

• The log-derivative trick

• The REINFORCE algorithm

• The variance problem

• The advantage function

• The actor-critic architecture

• Generalized advantage estimation (GAE)

• How LLMs are tuned with RL

• And a hands-on comparison of REINFORCE vs. actor-critic on CartPole showing the variance difference in real training curves.

Everything is covered from scratch, so no RL background is required.

You can read Part 7 of the course here →

Reinforcement Learning Nanodegree Part 7

Why care?

Techniques like RLHF, PPO, GRPO, and DPO show up in every model release announcement.

But the machinery underneath them is often treated as a black box.

• PPO becomes a library call.

• RLHF becomes a pipeline stage.

• GRPO becomes a config option.

All of them are built on policy gradients, advantage estimation, and actor-critic architectures.

The policy gradient theorem, the variance-bias tradeoff, GAE, and baselines are the actual moving parts that determine whether post-training succeeds or fails.

When things go wrong (through reward hacking, training instability, mode collapse), understanding these fundamentals is what lets you reason about why, instead of just tuning hyperparameters and hoping.

This series builds that layer of understanding from the ground up.

In the last seven chapters, we have covered bandits, MDPs, value functions, deep Q-learning, and now we have the policy gradient methods that power modern LLM alignment.

• Part 1: Foundations of Reinforcement Learning →

• Part 2: Markov Decision Processes and Value Functions →

• Part 3: Bellman Equations and Dynamic Programming →

• Part 4: Model-Free Learning in RL →

• Part 5: Function Approximation in RL →

• Part 6: Introduction to Deep RL and DQN →

• Part 7: Policy Gradients: REINFORCE and Actor-Critic →

Each concept builds on the last, and by this point, things like advantage-weighted policy updates feel like natural tools rather than abstract formulas.

👉 Over to you: What topics would you like us to cover in this RL series?

Thanks for reading!

What made Claude feel different from a normal app is that the agent could act inside the interface instead of only talking in a chat box.

For instance, Claude Artifacts let an agent render real UI, charts, dashboards, and interactive components that assemble live inside the response.

Every major AI product tried to replicate it.

But the problem was that, unlike reasoning, planning, tool-calling, etc., none of it shipped natively with LangGraph, CrewAI, or Google ADK.

So teams started building an owned version that required engineering the entire interface layer from scratch.

Most teams, however, just settled for shipping the agent as a backend API in a chat box since rendering the UI is only one piece of it.

To actually make it work, the interface layer also needed real-time streaming, state kept in sync between agent and UI, conversations that persist across sessions, and reconnection when a user refreshes mid-run.

CopilotKit (GitHub repo) is now the only open-source framework that actually lets you build your own full-stack Claude-like apps.

It decouples the agent from the interface, talking over AG-UI (an open protocol for agent-to-user communication).

Being a standard protocol, the frontend never needs to know whether it is talking to a LangGraph or a CrewAI agent. You can change the backend anytime, and the UI will never notice.

In practice, CopilotKit’s interface layer gives several pre-implemented React building blocks that wire the agent directly into the app, like:

• generative UI, so the agent renders real components instead of text

• chat windows, sidebars, and popups, or a fully headless setup

• shared state, so the agent and app stay in sync

• human-in-the-loop approvals, where the agent waits before acting

• persistent threads that store the whole session, including the agent-user interactions and generated UI, not just text

And because that full history is captured, those interactions can feed a self-learning layer that also improves the agent from real usage over time.

The interface layer that Anthropic spent years engineering in-house is now literally available to any developer/team.

CopilotKit is open-source with 30k+ GitHub stars, and AG-UI, the protocol underneath, is already supported across every major agent framework: LangGraph, CrewAI, Mastra, Google ADK, and more.

CopilotKit GitHub repo →

(don’t forget to star it ⭐ )

We are working on a hands-on demo for this. Stay tuned!

[Hands-on] Hermes agent desktop app

The Hermes Desktop App is insanely good.

It’s now the best way to run AI agents on your computer.

We recorded the full setup, start to finish.

Find it at the top.

Timestamps:

• 00:00 - Intro

• 00:46 - downloading and installing the desktop app

• 01:47 - picking your model and provider

• 02:52 - sessions, settings, and gateway connection

• 04:08 - adding a custom MCP server

• 04:58 - memory and context (the three-tier system)

• 06:52 - connecting to Telegram

• 09:45 - skills and tools

• 11:06 - the skills hub (built-in and community skills)

• 11:50 - creating a custom skill

• 14:18 - artifacts

• 15:11 - going from one to many agents (profiles and personas)

• 18:12 - agents working as a team (Hermes Kanban)

• 19:01 - outro

We also did a full walkthrough on Hermes Agent recently, covering Everything you need to understand and customize Hermes Agent, self-evolving skills, three-tier memory, GEPA optimization, and going from 1 to 10 agents that work 24/7.

Read it here →

Enjoy!

P.S. For those wanting to develop “Industry ML” expertise:

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?

• Drive revenue?

• Can you scale ML models?

• Predict trends before they happen?

We have discussed several other topics (with implementations) that align with such topics.

Develop "Industry ML" Skills

Here are some of them:

• Learn everything about MCPs in this ​crash course with 9 parts →​

• Learn how to build Agentic systems in a crash course with 14 parts.

• Learn how to build real-world RAG apps and evaluate and scale them in this crash course.

• Learn sophisticated graph architectures and how to train them on graph data.

• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.

• Learn how to run large models on small devices using ​Quantization techniques.

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using ​Conformal Predictions.

• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.

• Learn how to scale and implement ML model training in this practical guide.

• Learn techniques to reliably test new models in production.

• Learn how to build privacy-first ML systems using ​Federated Learning.

• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

Doc Holiday solves the documentation lag that ships with every release. Most teams treat docs as a post-ship task. In practice, it gets skipped. Changelogs go unwritten. Customer announcements are often held up for weeks. The support team ends up one release behind, permanently.

To eliminate the lag, Doc Holiday sits in the CI/CD pipeline. When a PR merges, it reads the commit history, linked tickets, and connected specs, then generates the changelog, release notes, and documentation updates. Outputs go to wherever the team already works: Confluence, Notion, Google Docs, Slack.

Documentation stays current as a byproduct of shipping, not as a separate effort.

Get started with Doc Holiday here →

Thanks to Doc Holiday for partnering today!

Researchers found a way to make LLMs 8.5x faster!

Speculative decoding is quite an effective way to address the single-token bottleneck in traditional LLM inference.

A small “draft” model first generates the next several tokens, then the large model verifies all of them at once in a single forward pass.

If a token at any position is wrong, you keep everything before it and restart from there. This never does worse than normal decoding.

But current drafters in Speculative decoding still guess one token at a time. That makes the drafting step itself a bottleneck, capping real-world speedups at 2-3x.

DFlash is a new technique that swaps the autoregressive drafter with a lightweight block diffusion model that guesses all tokens in one parallel shot.

We covered diffusion models from scratch, covering theory, math, why they work, training, inference and practical engineering in this course →

Drafting cost stays flat no matter how many tokens you speculate.

On top of that, the drafter is conditioned on hidden features pulled from multiple layers of the target model and injected into every draft layer, so it makes significantly better guesses than a drafter working from scratch.

In the side-by-side demo below, vanilla decoding runs at 48.5 tokens/sec. DFlash hits 415 tokens/sec on the same model, with zero quality loss.

It’s already integrated with vLLM, SGLang, and Transformers, with draft models on HuggingFace for several models like Qwen3, Qwen3.5, Llama 3.1, Kimi-K2.5, gpt-oss, and many more.

To learn more about Speculative decoding, we covered it here →

You can find the GitHub repo here →

KV caching is another must-know technique to boost LLM inference. We covered it here →

👉 Over to you: What use case are you working on that can benefit from this new technique?

Thanks for reading!

P.S. For those wanting to develop “Industry ML” expertise:

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?

• Drive revenue?

• Can you scale ML models?

• Predict trends before they happen?

We have discussed several other topics (with implementations) that align with such topics.

Develop "Industry ML" Skills

Here are some of them:

• Learn everything about MCPs in this ​crash course with 9 parts →​

• Learn how to build Agentic systems in a crash course with 14 parts.

• Learn how to build real-world RAG apps and evaluate and scale them in this crash course.

• Learn sophisticated graph architectures and how to train them on graph data.

• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.

• Learn how to run large models on small devices using ​Quantization techniques.

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using ​Conformal Predictions.

• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.

• Learn how to scale and implement ML model training in this practical guide.

• Learn techniques to reliably test new models in production.

• Learn how to build privacy-first ML systems using ​Federated Learning.

• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

AI tools typically split work and code into two products.

For instance:

• Claude Code lives in a separate interface from Claude.

• ChatGPT’s agent and Codex are different products.

On its own, that split looks harmless. But the cost shows up when a single task crosses the line, like a bug discussed in Slack that needs a code change.

Neither product carries the other's history, connectors, or task state, so you re-explain the same context twice, and each agent only ever sees half the work.

Mistral Vibe is taking an interesting approach by collapsing that into one agent with two modes.

• Work Mode handles long-horizon tasks across your inbox, calendar, and docs. It maps a plan, gets your sign-off, then executes across connectors like Google Workspace, Slack, and SharePoint. You hand off routine multi-step work in a single prompt, and every step stays visible as it runs.

• Code Mode takes a request from a prompt to a merged pull request. Sessions run in isolated sandboxes, in parallel, and end with a reviewable PR on GitHub.

The CLI is open-source under Apache 2.0, so you can fork it, swap models, or wire it into your own pipeline.

The model ships with open weights and self-hosts on as few as four GPUs, which means code never has to leave your infrastructure.

There’s also /teleport. Start a refactor in your terminal, hit /teleport, and the session moves to a cloud sandbox with its history, task state, and approvals intact. Close your laptop, let it keep running, and reattach later from any device.

Get started with Mistral Vibe here →

Find the CLI GitHub repo here → (don’t forget to star 🌟)

The evolution of retrieval

Google and Anthropic agree on one thing about RAG that retrieval is no longer a pipeline you build once.

Instead, it is a tool that something else calls.

• Anthropic’s MCP exposes retrieval as a tool an agent can call.

• Google now ships retrieval the same way, with its RAG Engine sitting under the Gemini agent platform alongside MCP servers.

That shift sounds small, but it changes the entire architecture underneath.

Today, let’s break down why classical RAG isn’t ideal in production, what a standing retrieval layer looks like, and how agents consume it.

Classical/Naive RAG

Naive RAG runs as a one-time pipeline inside a single app. A knowledge base gets chunked, embedded, and written to a vector DB.

At query time, the user’s query hits that vector DB for a similarity search, and the retrieved chunks go to the LLM.

This works when the answer lives in one corpus that rarely changes. It breaks on two fronts that show up immediately at scale.

The first is staleness. The embeddings reflect the source at the moment of indexing. When a doc changes in Notion or a row updates in Postgres, the vectors still point at the old content until someone reruns the pipeline.

The second is duplication. The retrieval logic is welded to one application. A second app, or an agent, that needs the same data has to rebuild the same connectors, chunking, and embedding from scratch.

Retrieval as a standing layer

The fix is to stop treating retrieval as a step inside the app and start treating it as a standing layer underneath it. A retrieval layer separates ingestion from query, and runs ingestion continuously instead of once.

On the ingest side, connectors pull from many sources, handle auth and extraction, chunk into entities, embed, and write to a vector store with a metadata store alongside it for source links and versions.

A sync process watches for changes and updates only what changed, using content hashing so unchanged data is not re-embedded.

On the query side, a request gets expanded, run through hybrid search that combines neural and keyword retrieval, then reranked before results come back with their source attribution.

The key property is that this layer is owned by no single app. It is exposed over an API, so a RAG chatbot and an autonomous agent can both query the same index.

How agents change the picture

An agent does not run retrieval once at the start.

It reasons, decides it needs context, calls search, reads the result, and decides whether to search again with a refined query. Retrieval becomes a tool inside a loop, not a fixed first step.

And the tool the agent calls is the retrieval layer.

The agent’s search function and the RAG app’s retrieval are the same query path into the same synced store.

They were never separate systems. Instead, one was just drawn inside the app and the other was pulled out and shared.

If you want to see this whole layer in practice, Airweave is an open-source implementation of it.

It connects to 50+ sources, continuously syncs them with content-hash incremental updates so the index stays current, and exposes everything through one search endpoint over both REST and MCP. The MCP endpoint is what lets an agent treat the whole layer as a single search tool.

You can find the repo here →

👉 Over to you: Is moving retrieval out of the app and into a shared layer the right call for your stack?

Thanks for reading!

P.S. For those wanting to develop “Industry ML” expertise:

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?

• Drive revenue?

• Can you scale ML models?

• Predict trends before they happen?

We have discussed several other topics (with implementations) that align with such topics.

Develop "Industry ML" Skills

Here are some of them:

• Learn everything about MCPs in this ​crash course with 9 parts →​

• Learn how to build Agentic systems in a crash course with 14 parts.

• Learn how to build real-world RAG apps and evaluate and scale them in this crash course.

• Learn sophisticated graph architectures and how to train them on graph data.

• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.

• Learn how to run large models on small devices using ​Quantization techniques.

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using ​Conformal Predictions.

• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.

• Learn how to scale and implement ML model training in this practical guide.

• Learn techniques to reliably test new models in production.

• Learn how to build privacy-first ML systems using ​Federated Learning.

• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

Datalab has just released Surya OCR, a state-of-the-art OCR model that scores 83.3% on the olmocr benchmark (top under 3B).

• 650M params

• supports 91 languages

• 5 pages/s on RTX 5090

• runs on CPU, GPU, MPS

• 83.3% olmocr bench score (top under 3B)

• Full layout information

• Extracts + captions images and diagrams

• Strong handwriting, math, form, and table support

Find the GitHub repo here → (don’t forget to star it ⭐️)

[Hands-on] Collapse CRM pipelines into one table

A table is a passive store. It persists rows and exposes reads and writes, but it has no execution model of its own.

Any logic that operates on a row runs in a separate process and writes the result back.

Enrichment pipelines are the clearest example of this, because once the table fills with data, everything useful has to happen outside it.

Take a leads table. The rows can be easily enriched with emails, headcount, and tech stack.

But the mechanisms to score a lead and trigger a follow-up run elsewhere. This requires standing up a service, registering a webhook, and writing a sync job to land the output back on the right row.

Of course, the scoring logic is trivial. But the integration and glue around it is not. And it multiplies per step, since the follow-up adds another service, another webhook, another write path back into the table.

A smart implementation of this now actually exists in Sim (GitHub Repo), which is an open-source (28k+ stars) visual AI workspace for building and running agentic workflows. You can self-host it on Docker.

It has three core pieces.

• A visual canvas where you connect blocks into a workflow.

• Tables that store data and keep it across runs.

• Mothership, a natural-language layer that creates tables, wires up blocks, and builds workflows from a plain-English description.

Tables hold typed columns and expose more than ten operations as workflow blocks, like query, insert, upsert, and update.

The new part is that a column can also run a workflow, not just store a value.

Instead of holding a static value, a column can run enrichment or a full workflow against its row, then write the output back into the same row. The execution model that was missing from the table now lives on the table.

So there’s no external service to stand up, a webhook to register, or a sync job to reconcile, because the logic runs where the data already is.

A column can also watch another column and fire when its value changes, so the steps that used to be glued together with webhooks become a chain inside one table.

Let’s walk through how its enrichment columns and workflow columns work, and how one column can trigger the next.

Enrichment columns

An enrichment column lives on the table itself. There is no separate tool to export the rows to and import them back from.

You add it from the column menu, or you describe it to Mothership and let it set up the column. In the demo, Mothership suggests enriching a CRM table with work emails and company data, and the columns fill in row by row.

A few things make this practical day to day.

• Each row shows its own live status, so partial results appear as they land instead of in one batch at the end.

• The column header configures the output fields. A column can show headcount and hide the raw LinkedIn URL.

• Several enrichment columns run on the same table at once.

• Every result writes back in place, with no re-import and no sync job.

The demo says Sim pulls from the same data sources Clay does. Treat that as a vendor claim until you test it on real records. The point that holds either way is that the enriched data lands in the table you already work in.

Workflow columns

Enrichment is the smaller idea. The workflow column is where the table starts doing real work.

A workflow in Sim is a chain of blocks. A block can be an AI model call, a database write, a Slack message, or a condition. You build the workflow once on the canvas, then add it as a column type on a table.

When a workflow runs as a column, it takes the full row as input. The column titles and their values get passed in, the workflow runs, and the outputs you pick get written back into columns on the same row.

In the demo below, the workflow scores each lead. It passes the company name and description in, runs an AI step against target-customer criteria, and writes back two columns, a short reasoning summary, and a qualified yes/no.

You can also describe the workflow you want by chatting with the table, and Mothership builds it and adds it as a column. That is the part that saves the wiring. The logic that would normally live in a separate tool gets generated from a sentence and dropped onto the table.

You can gate the run with auto-run conditions. In the demo, the scoring workflow fires only once both the industry and description are filled. Rows still waiting on enrichment never trigger, so no model calls are wasted on incomplete data.

Chaining columns

A column’s output can trigger another workflow. One column watches another, and when its value changes, the next workflow runs.

In the demo below, we wire this as an approval gate. The scoring column writes its reasoning and talking points, a reviewer reads them and flips an “approved” field, and that change triggers the next workflow on that row.

Check this out:

That gives you a full pipeline inside one table. Enrichment fills the row, an AI step scores it, a human approves it, and the approval kicks off the next action, with each step reacting to the column before it.

When is this worth it?

Running logic as columns pays off when a job has several steps that depend on each other. Enrichment feeds scoring, scoring feeds an action, and each step needs the previous one to finish on that exact row. Keeping all of it on the table removes the glue that usually holds those steps together.

For a single step, it is overkill. Filling in a missing email needs one action and no coordination, and a spreadsheet with one API call is the simpler choice. The value shows up on a chain, which is exactly where webhooks and sync jobs normally pile up.

Taking it further

The last step in the demo points to something more. An approved row triggers a workflow that calls Mothership as a block, and Mothership builds a brand-new workflow in the workspace, shaped around that one lead.

The output of the pipeline is itself a workflow. Instead of ending at a fixed action, the table generates the next piece of automation from the row that triggered it. It is a single demo moment rather than a proven pattern, but it hints at a table that doesn’t just run a pipeline, it assembles one.

To try it out yourself...

Sim is open-source and self-hostable. The repo is at simstudioai/sim (28k+ stars 🌟, Apache 2.0).

Building the full pipeline from scratch takes around 5 minutes. You describe what you need, and Mothership creates the tables, configures the columns and conditions, and adds the workflows. There is no integration code to write by hand.

The repo is here →

The Mothership and Tables docs are here →

👉 Over to you: which pipeline would you collapse into a table first, lead scoring, support ticket routing, or content personalization?

Thanks for reading!

We gave Claude Code one prompt, and it built a full-stack weather intelligence dashboard.

It’s an interactive 3D globe with a day/night cycle using NASA satellite imagery, city lights on the dark hemisphere, weather icons that switch between sun and moon based on local time, and a time travel slider that scrubs through 10 days of data, including a 3-day forecast.

The backend runs on Tiger Cloud by Tiger Data, which gives you managed TimescaleDB on Postgres.

Tiger Cloud Platform

Claude Code provisioned the database, created the schema, seeded the data, and built the frontend in a single session. Let’s walk through the setup, the prompt, and what we built.

Setting up the backend

The entire database infrastructure was provisioned from inside Claude Code using the Tiger CLI MCP server. These are three commands to get started:

# Install Tiger CLI
curl -fsSL https://cli.tigerdata.com | sh

# Authenticate with Tiger Cloud
tiger auth login

# Connect Claude Code to Tiger Cloud via MCP
tiger mcp install claude-code

After restarting Claude Code, it has full access to Tiger Cloud through the MCP server: creating services, running SQL, and querying documentation.

The prompt

The prompt we gave Claude Code covered the full stack in one shot:

Build a real-time weather intelligence dashboard with an interactive
3D globe tracking cities worldwide.

Backend: Use our Tiger Cloud account (via Tiger MCP) to create a
TimescaleDB service. Create a hypertable for hourly weather readings
(city, temperature, humidity, wind speed, weather condition, timestamp).
Set up continuous aggregates for hourly and daily rollups. Seed
realistic weather data for 7 days across 150 cities.

Frontend: Next.js + Three.js. Dark theme. The globe should have
a day/night cycle using NASA satellite imagery, city lights on the
dark side, weather icons (sun/moon based on local time), and a time
travel slider spanning 10 days (7 historical + 3 forecast). Add a
city detail panel with temperature sparklines when clicking a city.

After this, Claude Code provisioned the TimescaleDB service, created the hypertable, set up continuous aggregates, seeded 25,000+ rows of weather data, generated the forecast table, and built the complete Next.js frontend with the Three.js globe.

The time travel slider queries thousands of rows on every position change.

On a regular Postgres table, this would require manual partitioning and index tuning to stay fast as data grows.

TimescaleDB partitions the data by timestamp automatically, so each query only hits the relevant time chunk. Continuous aggregates serve the trend charts and forecast layer from pre-computed rollups instead of rescanning raw data on every request.

To get started:

# 1. Sign up for Tiger Cloud (link below)

# 2. Install Tiger CLI
curl -fsSL https://cli.tigerdata.com | sh

# 3. Authenticate
tiger auth login

# 4. Connect Claude Code
tiger mcp install claude-code

# 5. Give Claude Code a prompt and let it build

Tiger Cloud gives new users $1,000 in free 30-day credit, no credit card required. That’s enough to run a production TimescaleDB instance for a full month and build whatever you want on top of it.

You can sign up for Tiger Cloud here →

Tiger Cloud Platform

Tiger CLI is open-source (Apache 2.0) and works with Claude Code, Cursor, Codex, Gemini CLI, and VS Code.

👉 Over to you: what would you build if Claude Code could provision your database infrastructure?

Thanks to Tiger Data for working with us on today’s issue!

Claude Code dynamic workflows, explained!

Anthropic recently release Opus 4.8, and everyone is talking about the benchmarks, the honesty improvements, and the cheaper fast mode.

But the feature that shipped alongside it might matter more for how we actually build: Dynamic Workflows in Claude Code.

Here’s what they are, how they differ from subagents and agent teams (which already existed), and why they change the game for large-scale agentic coding.

What are dynamic workflows?

Claude Code already had two multi-agent primitives before this release.

• Subagents are lightweight workers spawned from a main session. They do a focused task and report back. But they can’t talk to each other, and the main agent still acts as the bottleneck for orchestration. Every result routes through one context window.

• Agent Teams (shipped with Opus 4.6) removed that constraint. Multiple Claude instances coordinate through a shared task list and message each other directly. But they top out at 3-5 teammates practically, sessions don’t survive interruptions (if Claude crashes mid-task, the team is gone), and you still need to design the orchestration upfront.

We published an article on both of the above. You can read it here →

Dynamic Workflows sit above both.

Instead of Claude holding the plan in its context window, it writes a JavaScript orchestration script. That script becomes the plan. A JS runtime executes it, fanning work across tens to hundreds of parallel subagents automatically.

You describe the task. Claude decides how to split it, how many agents to spawn, how to verify results, and what to report back. The orchestration logic moves from the LLM’s memory into executable code.

Claude’s context window only ever sees the final converged answer. Not the intermediate results of hundreds of steps.

This is helpful because of several reasons:

• Scale → You can run up to 16 concurrent agents and 1,000 total per workflow, while Subagents max out at a handful and Agent Teams get messy past 5.

• Adversarial verification → This allows agents to tackle a problem from independent angles, other agents try to refute their findings, and the system iterates until answers converge. Compare this to subagents which just report back and Agent Teams collaborate but don’t adversarially verify.

• Resumability → You can save progress saves continuously. So interrupted jobs pick up where they left off while Agent Teams die with the session.

• Zero orchestration burden → You just describe the goal and Claude decides how to split work, how many agents to spawn, and how to verify results.

These are some best practices:

→ Start with a scoped task to calibrate token usage before going full-scale. Workflows consume significantly more tokens than a typical session.

→ Enable auto mode so Claude decides when a workflow is appropriate vs. when a simpler approach works.

→ Use the ultracode setting (effort menu) to let Claude auto-trigger workflows. This also sets reasoning to xhigh.

→ Review the execution plan on first trigger. A poorly scoped prompt will fan out agents unnecessarily.

→ Enterprise plans have workflows off by default. An admin needs to enable them.

As further reading, we published an article on Agent Teams and Subagents recently. You can read it here →

Thanks for reading!

Google’s back with another free 5-day AI Agents course.

The last edition hit 1.5 million learners, and this time, they will dive into vibe coding with agents.

The daily progression actually mirrors how agent development works in practice:

• Day 1: Agent fundamentals and vibe coding: You will go from basic chatbots to autonomous agents using natural language as the primary dev interface.

• Day 2: Tool integration and multi-agent systems: You will learn how to wire agents to external APIs and get multiple agents coordinating with each other reliably.

• Day 3: Context engineering: You will build agents that maintain reasoning across sessions using short-term recall, long-term memory, and persistent state.

• Day 4: Evals and security: You will set up testing, guardrails, and observability to catch the failure modes that only show up when agents have real tool access.

• Day 5: Prototype to production: You will deploy everything to production and build a capstone project where you ship your own agent end-to-end.

Everything runs on Kaggle notebooks with daily livestreams, taught by Google’s ML engineers.

1-2 hours a day and completely free to enroll.

Register before June 2nd here →

Register for Google's free course

Thanks to Google Cloud for partnering with us today!

Introduction to Deep RL and DQN

Part 6 of the RL series is here.

Reinforcement Learning Nanodegree Part 6

The previous chapter showed that tables and even linear features hit a ceiling.

This one covers what happens when you replace them with neural networks, what breaks immediately, and the two engineering choices that made deep value-based RL actually work.

You can read Part 6 of the course here →

It covers:

• The shift from linear to neural function approximation

• Why naive deep Q-learning fails

• How experience replay decorrelates training data

• How target networks stabilize learning

• The complete DQN algorithm that combines all of these

• And a from-scratch PyTorch implementation trained on CartPole with training curves and analysis.

Everything is covered from scratch, so no RL background is required.

You can read Part 6 of the course here →

Why care?

Look at what has happened in the past two years.

• DeepSeek-R1 used GRPO for reasoning.

• ChatGPT was shaped by RLHF.

• Claude uses constitutional AI with RL.

Every frontier LLM released recently has some form of reinforcement learning in its post-training pipeline.

RL is no longer a niche subfield for robotics and game-playing. It is a core component of how the most capable AI systems are built today.

Google Trends reflects this.

Search interest for “reinforcement learning” was nearly flat from 2004 to 2024. In the past year, it has gone vertical, hitting an all-time high.

The demand for RL expertise has followed.

If you look at ML engineering roles at labs like OpenAI, Anthropic, DeepMind, or any team working on post-training, alignment, or agentic systems, RL fluency shows up as a requirement consistently.

Understanding how reward signals shape model behavior, how policy optimization works, and how exploration interacts with credit assignment is becoming as fundamental as understanding backpropagation was five years ago.

This series is structured the same way as our MLOps/LLMOps course: concept by concept, with clear explanations, diagrams, math where it matters, and hands-on implementations you can run.

• Part 1: Foundations of Reinforcement Learning →

• Part 2: Markov Decision Processes and Value Functions →

• Part 3: Bellman Equations and Dynamic Programming →

• Part 4: Model-Free Learning in RL →

• Part 5: Function Approximation in RL →

• Part 6: Introduction to Deep RL and DQN →

👉 Over to you: What topics would you like us to cover in this RL series?

Thanks for reading!

Transformer Lab is an open-source ML platform that orchestrates GPUs across any cloud and runs any training or eval workflow you define:

• supports LoRA, QLoRA, DPO, ORPO, SIMPO

• use it from a GUI, CLI, or agent skill.

• works with MLX, vLLM, Ollama, and HF Transformers

• one-click convert between HF, GGUF, and MLX

• diffusion training and inference built in

• LLM-as-a-judge evals plus the EleutherAI harness

• submits jobs to Slurm and SkyPilot from the same UI

• runs on Apple Silicon, NVIDIA, and AMD

The same interface a solo researcher uses on a MacBook also drives a 64-GPU cluster. Nothing else does that today.

Find the GitHub repo here → (don’t forget to star it ⭐️)

Transformer Lab GitHub repo

Why agent crashes are nothing like database crashes

When a database crashes and restarts, it replays the write-ahead log and reconstructs the exact same state, down to every transaction, row, and index entry.

This deterministic recovery is a solved problem in databases and has been for decades.

Agent crashes don’t work this way, and just restarting it is not the ideal option.

Let’s dive in to understand the problem and also see how Google Cloud’s new Agent Platform has engineered the infra to solve it.

The decision drift problem

Consider an agent that processed financial records over five days.

• On day 1, it ingested 4,000 records and normalized formats. One record has an ambiguous date field (“03/04/2026”). The agent interprets it as March 4th. Every downstream decision, the cross-referencing on day 2, the anomaly detection on day 3, builds on that interpretation.

• On day 3, the agent crashes.

A stateless restart would involve re-ingesting all 4,000 records from scratch.

Now, the LLM may interpret that same ambiguous date as April 3rd.

That single different decision influences every subsequent step. This is what makes agent state fundamentally different from database state.

Checkpoint-and-resume strategy for Agents

The fix is conceptually simple.

The agent should periodically save its intermediate state, including progress, accumulated decisions, and the reasoning chain that led to them. If it crashes, it reloads the last checkpoint and continues with every prior decision intact.

Technically, here’s what you’d need to do:

• Implement persistent state serialization across sessions

• Manage storage and retrieval

• Reconstruct the agent’s full context window exactly as it was before the crash

• Handle edge cases like human-in-the-loop approvals where the agent needs to pause indefinitely without consuming compute.

If you want to see it in practice, Google Cloud’s Agent Platform now actually ships this as a native platform capability through three mechanisms that work together.

• Memory Bank gives agents a persistent state that accumulates across sessions. So the agent on day 5 can read what it decided on day 1 without re-processing. Under the hood, it uses scoped memory extraction, converting conversation history and intermediate decisions into searchable memories that persist and consolidate automatically.

• Resume Agents handle checkpoint-and-resume natively within the ADK. When an agent is interrupted (due to a crash, timeout, or human approval gate), Agent Runtime saves the full state. When it resumes, the context reloads exactly as it was. In the ADK, a basic resumable agent definition looks like this:

from google.adk import Agent

root_agent = Agent(
    model="gemini-3.1-flash-lite-preview",
    name="reconciliation_agent",
    description="Reconciles financial records across systems.",
    instruction="""Process records in batches. After each batch,
    save progress to state. If resumed after interruption,
    check state for last completed batch and continue from
    there.""",
    tools=[ingest_records, cross_reference, flag_anomalies],
)

• Ambient Agents make execution event-driven. Instead of waiting for a user prompt, the agent activates when new data arrives. For instance, an invoice processing agent can trigger on incoming records, not on a human typing a query.

Why care?

The deeper point here isn’t about Google’s specific implementation.

It’s that the industry has been treating agent memory as a retrieval problem (RAG, vector search, context windows) when the harder problem is actually a consistency problem.

An agent that processes data over days needs the same guarantees that databases have provided for decades, i.e., if it crashes, the recovered state must be identical to the pre-crash state.

LLM non-determinism makes that impossible without explicit checkpointing.

Agent Platform is the first infrastructure that treats this as a platform-level concern rather than leaving it to application developers to solve with custom Redis pipelines and state serialization code.

You can try the Agent Platform here →

👉 Over to you: what’s the longest-running agent workflow you’ve tried to build, and where did state management bite you?

Thanks for reading, and to Google Cloud for working with us on today’s issue!

AWS Summit India Online is happening on June 3rd, and it’s completely free.

The part we like about this event is that it hosts hands-on virtual labs with Amazon Bedrock and agentic AI. These are not just talks about agents, but actual labs where participants can experiment with them to build AI systems in production.

Beyond the labs, there are 50+ sessions covering agentic AI, serverless computing, cloud modernization, and real-world case studies from companies across industries. Live Q&A with AWS solution architects and AWS Partners is available throughout the day.

The event runs from 10 AM to 4 PM IST, and on-demand replays stay accessible through July 3rd, so it’s easy to learn at your own pace from anywhere.

Register here (for free) →

Register for free here!

Thanks to AWS for partnering today!

RAG vs. Graph RAG vs. Agentic RAG

Standard RAG embeds documents into vectors and retrieves the most similar chunks via similarity search. For direct factual lookups, this works well.

But it breaks down when a query needs to connect facts spread across multiple documents. Similarity search retrieves individual chunks, not the relationships between them.

Graph RAG adds a knowledge graph layer on top.

• During indexing, an LLM extracts entities and relationships from the documents.

• During retrieval, the system traverses these connections instead of relying on embedding similarity alone.

This is what enables multi-hop queries.

Say a vector DB stores three facts about internal services:

↳ “The checkout service uses payments API.”

↳ “The payments API runs on cluster-3.”

↳ “Cluster-3 is scheduled for maintenance on Friday.”

Someone asks: “Will the checkout service be affected by Friday’s maintenance?”

Vector search can likely retrieve facts 1 and 3 because the query mentions “checkout service” and “Friday maintenance.”

But it will miss fact 2, which connects the payments API to cluster-3.

That middle fact sits too far from the query in embedding space. It mentions neither “checkout” nor “maintenance,” so it never makes it into the retrieved context.

A knowledge graph connects these as linked entities, and graph traversal finds the full path in one query.

Agentic RAG takes a different approach entirely.

Instead of a fixed retrieval pipeline, an LLM agent decides at query time which tools to invoke, which sources to query, and in what order.

Check the visual below again to understand the three architectures thoroughly.

One thing to note here is that these three aren’t levels of sophistication that you need to graduate through.

Instead, they solve different query types.

↳ Single-hop factual lookups → standard RAG

↳ Multi-hop relationship queries → Graph RAG

↳ Dynamic multi-source tasks with tool use → Agentic RAG

If you want to dive further, our full RAG crash course discusses RAG from basics to beyond:

• ​​​​RAG fundamentals​​​​

• ​​​​RAG evaluation​​​​

• ​​​​RAG optimization​​​​

• ​​​​Multimodal RAG​​​​

• ​​​​Graph RAG​​​​

• ​​​​Multivector retrieval using ColBERT​​​​

• ​​​​RAG over complex real-world docs ft. ColPali​​​​

👉 Over to you: Which RAG architecture are you running in production?

Thanks for reading!

When something breaks in production, the fix is usually simple. The hard part is the 30 minutes before it, jumping between Datadog, GitHub, Cloud Run logs, and Slack history, trying to figure out what changed and why.

Claude Code can’t help here. It has no visibility into APM traces, ticket history, or what the team decided last month.

​CodeRabbit Agent​ sits inside Slack and connects to your code, tickets, docs, monitoring stack, and cloud infra. You mention it in a thread, and it pulls traces, cross-references recent PRs, opens a targeted fix, and files the postmortem. The whole incident stays in one thread where the team sees every step.

It also retains what the team decided across conversations, so the same context doesn’t need to be re-derived the next time something breaks.

New workspaces receive $50/user in free agent minutes.

​Get started with CodeRabbit Agent →​

Thanks to CodeRabbit for partnering today!

[Video] Hermes agent masterclass

We recently released a full issue on the Hermes Agent, and many of you reached out to publish it as a video guide.

We just put together one and you can find the video at the top.

This 48 mins video covers everything you need to understand and customize Hermes Agent.

Self-evolving skills, three-tier memory, GEPA optimization, and going from 1 to 10 agents that work for you 24/7.

Enjoy.

Thanks for reading/watching.

P.S. For those wanting to develop “Industry ML” expertise:

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?

• Drive revenue?

• Can you scale ML models?

• Predict trends before they happen?

We have discussed several other topics (with implementations) that align with such topics.

Develop "Industry ML" Skills

Here are some of them:

• Learn everything about MCPs in this ​crash course with 9 parts →​

• Learn how to build Agentic systems in a crash course with 14 parts.

• Learn how to build real-world RAG apps and evaluate and scale them in this crash course.

• Learn sophisticated graph architectures and how to train them on graph data.

• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.

• Learn how to run large models on small devices using ​Quantization techniques.

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using ​Conformal Predictions.

• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.

• Learn how to scale and implement ML model training in this practical guide.

• Learn techniques to reliably test new models in production.

• Learn how to build privacy-first ML systems using ​Federated Learning.

• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

• Free course to learn Generative UI, Anthropic’s most viral feature.

• ​Claude Code used 3x fewer tokens with just one change!​

• Docker explained in 2 minutes!

• Active learning in ML.

Learn Generative UI, Anthropic’s most viral feature​

Andrew Ng’s DeepLearningAI just ​released​ a new free course on building Interactive Agents with Generative UI, covering the full stack from scratch:

Claude Artifacts showed what happens when agents generate UI instead of text, like charts, dashboards, and interactive components, all assembled live inside the chat.

Every major AI product tried to replicate it.

CopilotKit is the only open-source framework (​repo​) that actually lets you build your own full-stack Claude-like apps.

The course covers three approaches on the Generative UI spectrum, including:

1) Controlled → the agent picks from pre-built components like pie charts and flight cards.

2) Declarative → the agent assembles layouts from reusable building blocks using A2UI, co-developed with Google.

3) Open-ended → the agent generates arbitrary HTML/SVG from scratch, streamed token-by-token into a sandboxed iframe.

You can implement the entire Generative UI spectrum through AG-UI, and it works out of the box with LangGraph, Google ADK, CrewAI, Mastra, AWS Strands, and more.

This entire stack is built on top of CopilotKit.

31k+ GitHub stars, with SDKs for React, Next.js, Angular, and Vue.

​You can watch the course here →​

​And here’s the CopilotKit GitHub repo →​

Devs who watch this tonight will know how to build agent interfaces that go beyond plain text by tomorrow.

​Claude Code used 3x fewer tokens with just one change!​

Before writing a single line of code, the agent must know:

• the existing tables and schemas

• the configured auth providers

• the available storage buckets

• the deployed edge functions

• the available AI models

• the active RLS policies

Most backends expose this via MCP tools, which means the agent discovers each piece through a separate call.

This is a problem because Claude Code re-sends the full conversation on every turn, so each discovery call is more expensive than the last.

So by the time the agent understands the backend, a significant chunk of the token budget is already gone.

Some info, like auth provider config, isn’t queryable at all. Either the developer manually provides it in the prompt or the agent assumes it.

These wrong assumptions only surface as runtime errors after the code is already written.

And when something breaks, the errors don’t say where the failure came from. So the agent retries, re-sending the entire growing conversation each time.

A smarter model has no way to skip these gaps.

If anything, it actually tries harder to fill them, which is why MCPMark V2 benchmarks showed a ~60% increase in backend token usage after a model upgrade.

In our own test, this setup cost 10.4M tokens and 10 manual interventions on a single RAG app.

But we brought that down to 3.7M tokens and 0 manual interventions by using InsForge (​GitHub repo​) as the backend context engineering layer for Claude Code (open-source and self-hostable via Docker).

It provides the same primitives as Supabase but structures the entire information layer for agents instead of dashboards.

In one CLI call that consumes ~500 tokens, the agent can see the full backend topology before writing a single line of code.

This includes auth, database, storage, edge functions, model gateway, micro VMs, and deployment.

Also, instead of loading the entire product surface into context on every task, four narrowly scoped skills activate only when relevant to keep cognitive load minimal.

And to ensure efficient retries if needed, every CLI operation returns structured JSON with meaningful exit codes, so the agent never has to guess what to do next.

​Here’s the InsForge GitHub Repo →​ (don’t forget to star it ⭐)

​You can find our full 100% local setup guide in this issue →​

Docker explained in 2 minutes!

Developers typically use Docker daily without understanding what happens under the hood. Here’s everything you need to know.

Docker has 3 main components:

• Docker Client: Where you type commands that talk to the Docker daemon via API.

• Docker Host: The daemon runs here, handling all the heavy lifting (building images, running containers, and managing resources)

• Docker Registry: Stores Docker images. Docker Hub is public, but companies run private registries.

Here’s what happens when you run “docker run”:

• Docker pulls the image from the registry (if not available locally)

• Docker creates a new container from that image

• Docker allocates a read-write filesystem to the container

• Docker creates a network interface to connect the container

• Docker starts the container

That’s it.

The client, host, and registry can live on different machines. This is why Docker scales so well.

Understanding this architecture makes debugging container issues much easier. You’ll know exactly where to look when something breaks.

Active learning in ML​

Data annotation is difficult, expensive, and time-consuming.

Active learning is a relatively easy and inexpensive way to build supervised models when you don’t have annotated data to begin with.

As the name suggests, the idea is to build the model with active human feedback on examples it is struggling with.

The visual below summarizes this:

1. Manually label a tiny percentage of the dataset.

2. Build a model on this small labeled dataset (model should be able to provide confidence values)

3. Generate predictions on the unlabeled dataset.

4. Label low confidence predictions and add them to the labeled dataset.

5. Back to step 2.

Active learning is a huge time-saver in building supervised models on unlabeled datasets.

The only thing that you have to be careful about is generating confidence measures.

If you mess this up, it will affect every subsequent training step.

​To learn more techniques on supercharging ML model training (with implementation), read this →​

Thanks for reading!

Strands Agents, the open source agent harness SDK from AWS gives you a model-driven way to build production-ready agents in just a few lines of code.

Any model, any cloud. You get context management, execution limits, and observability before writing a line of config.

Swap backends when you scale, and your code stays the same.

Hooks let you intercept any step in the agent loop to log, validate, or redirect it. Guardrails catch mistakes before they run.

Build your first agent harness →

Thanks to Amazon Web Services (AWS) for partnering today!

The No. 1 Deep Researcher Beats Claude and ChatGPT Using a Counterintuitive Trick

The No. 1 deep researcher beats Claude and ChatGPT with a trick neither uses.

We studied the open-source architecture behind it.

A counterintuitive thing we found is that the orchestrator agent that runs the entire research strategy has no search access.

It can’t query the web or open URLs.

This looks wrong at first glance. Every other deep research system gives its coordinator far more capability.

For instance:

• OpenAI’s approach trains a single model for many consecutive tool calls. It searches, reads, reasons, and writes the report in a long sequential chain.

↳ The researchers behind the No. 1 system (Onyx; GitHub Repo) observed that this causes the model to spend cycles on low-value searches instead of maintaining a high-level research strategy.

• Anthropic and Google use an orchestrator-researcher pattern similar to Onyx’s system. The key difference is how aggressively Onyx constrains the orchestrator.

Most orchestrators have access to search and retrieval tools alongside dispatch capabilities. And the moment an orchestrator can search, it will.

So instead of decomposing a query into focused research threads, it starts answering the question itself.

It pulls a few results, skips proper task decomposition, and produces a surface-level report from whatever it found first.

Stripping search from the orchestrator forces it to write self-contained and coherent task briefs for each research agent.

The researchers also kept the architecture only two levels deep. When info passes through multiple agents, each one subtly distorts it through summarization/reinterpretation. Keeping it to two levels prevents this.

These two constraints sit inside a larger three-phase pipeline (the visual below maps this):

• Phase 1 decomposes the query into up to 6 research directions. No tool access prevents the model from prematurely answering.

• Phase 2 dispatches 3 isolated research agents. Each runs up to 8 sub-cycles of search, read, and think, to produce an intermediate report with citations. The agents can also search internal enterprise docs (Confluence, Slack, 100+ connectors) with document-level permissions enforced, unlike proprietary solutions.

• Phase 3 runs a deterministic step that renumbers and deduplicates to produce a report with a unified citation map.

This pattern has been ranked No. 1 on DeepResearch Bench. The whole implementation is available on GitHub, and you can try it yourself.

Here’s the Onyx Repo → (don’t forget to star it ⭐ )

Onyx GitHub Repo

👉 Over to you: Which one is your go-to deep researcher right now?

Thanks for reading!

Reinforcement Learning Nanodegree part 5

Everything we built so far (value functions, Bellman equations, MC, TD, SARSA, Q-learning) assumed you could store one value per state in a table.

This chapter covers what happens when that assumption breaks, which it does in every real problem.

It covers:

• Why lookup tables stop working for real-world problems

• How to replace them with parameterized functions that generalize across similar states.

• The learning algorithms that make this work (gradient Monte Carlo and semi-gradient TD)

• What happens when you combine function approximation with bootstrapping and off-policy learning

• and a full hands-on implementation that trains an agent to solve Mountain Car, a continuous-state control problem where the car has to learn to build momentum by swinging back and forth.

Everything is covered from scratch, so no RL background is required.

You can read Part 5 of the course here →

Why care?

Look at what has happened in the past two years.

• DeepSeek-R1 used GRPO for reasoning.

• ChatGPT was shaped by RLHF.

• Claude uses constitutional AI with RL.

Every frontier LLM released recently has some form of reinforcement learning in its post-training pipeline.

RL is no longer a niche subfield for robotics and game-playing. It is a core component of how the most capable AI systems are built today.

Google Trends reflects this.

Search interest for “reinforcement learning” was nearly flat from 2004 to 2024. In the past year, it has gone vertical, hitting an all-time high.

The demand for RL expertise has followed.

If you look at ML engineering roles at labs like OpenAI, Anthropic, DeepMind, or any team working on post-training, alignment, or agentic systems, RL fluency shows up as a requirement consistently.

Understanding how reward signals shape model behavior, how policy optimization works, and how exploration interacts with credit assignment is becoming as fundamental as understanding backpropagation was five years ago.

This series is structured the same way as our MLOps/LLMOps course: concept by concept, with clear explanations, diagrams, math where it matters, and hands-on implementations you can run.

• Part 1: Foundations of Reinforcement Learning →

• Part 2: Markov Decision Processes and Value Functions →

• Part 3: Bellman Equations and Dynamic Programming →

• Part 4: Model-Free Learning in RL →

• Part 5: Function Approximation in RL →

👉 Over to you: What topics would you like us to cover in this RL series?

Thanks for reading!

Onyx is a self-hostable AI chat platform that works with any LLM, like Claude, GPT, Gemini, Llama, or any open-weight model you want.

Onyx GitHub Repo

Here’s what it ships with:

• Agents that chain multiple tools in sequence

• RAG with full indexing across 40+ connectors (Slack, Drive, Confluence, Jira, GitHub, email, call transcripts)

• Deep research ranked #1 on DeepResearchBench, above every proprietary alternative

• MCP support for connecting to external systems

• Code interpreter for data analysis and file generation

• Self-host on your own infrastructure via Docker in a few mins

Unlike Claude’s MCP-based connectors that query your tools at runtime, Onyx actually indexes all your data. That means faster, more reliable search across everything your team has ever written.

The entire code is open-source under the MIT license, so you can see the full implementation on GitHub and try it yourself.

Find the GitHub repo here → (don’t forget to star it ⭐️)

Agent memory is only as good as its schema

When you give an agent a knowledge graph for memory, the default behavior is that the LLM handling extraction decides the structure on its own.

It picks the entity types, the relationship labels, and the attributes.

The results are generic.

Everything becomes a Topic or Object and every connection is labeled RELATES_TO, resulting in a graph that can’t be queried meaningfully.

Zep’s Graphiti (open-source with 26k+ GitHub stars) fixes this with a prescribed ontology.

You can define entity types and edge types as Pydantic models, and the extraction model classifies against these definitions instead of guessing.

Let’s look at why this matters, how the extraction pipeline works, with code!

Why flat retrieval breaks on multi-hop reasoning

Vector-based memory stores facts as text chunks and retrieves them by semantic similarity. That works until a query requires connecting facts that don’t appear in the same chunk.

Consider three facts stored about a project.

• Alice manages Project Atlas

• Project Atlas runs on PostgreSQL

• The PostgreSQL cluster went down Tuesday

A query like “was Alice’s project affected by Tuesday’s outage” needs all three.

Vector search will retrieve just facts 1 and 3 because both mention relevant terms. Fact 2 is the bridge connecting Alice to PostgreSQL through Project Atlas, but it mentions neither Alice nor Tuesday. Similarity search misses it.

A knowledge graph stores entities as nodes and relationships as edges. Instead of matching text, it traverses connections.

That chain (Alice → manages → Project Atlas → runs on → PostgreSQL) is what makes multi-hop reasoning work, and it is invisible to flat vector retrieval.

Where extraction fits in the pipeline

Every graph-based memory system follows the same pipeline. LLM accepts the raw data and extracts entities and relationships.

These get stored as nodes and edges, and at query time, the system searches the graph and injects relevant facts into the agent’s prompt.

The extraction step determines everything downstream. It decides what the graph contains, how it’s structured, and what can be queried.

When extraction is unstructured, the LLM picks entity types and relationship labels on its own.

A developer conversation about building a web app called Nexus with Python, TypeScript, React, and Docker produces nodes labeled Topic and Object with RELATES_TO edges across the board.

Two things break without proper structure:

• Retrieval collapses to semantic similarity because it can’t filter by type when everything shares the same label.

• Domain constraints disappear because there are no structured attributes like status or category to distinguish “active vs. completed” or “frontend framework vs. database.”

Defining the schema with Pydantic

The fix is the same pattern used everywhere in the AI stack.

• FastAPI endpoints get Pydantic response models.

• Function calling tools get Pydantic schemas.

• Agent memory works the same way in Zep.

Define custom entity types using EntityModel (a subclass of Pydantic’s BaseModel) with EntityText fields and descriptions that guide the extraction model.

from zep_cloud.external_clients.ontology import EntityModel, EntityText
from pydantic import Field

class Project(EntityModel):
    """
    Represents a specific software project, application, 
    or codebase that the user is building or contributing to.
    """

    project_status: EntityText = Field(
        description="Current status: active, completed, paused, or archived.",
    )
    project_type: EntityText = Field(
        description="Type of project: web app, mobile app, API, CLI tool, etc.",
    )

The docstrings and field descriptions are important here because good descriptions with concrete examples give the extractor enough signal to classify accurately.

The Pydantic descriptions above aren’t just classification instructions. They teach the extractor vocabulary it doesn’t know.

A Technology entity follows the same pattern.

class Technology(EntityModel):
    """
    Represents a programming language, framework, library, 
    database, or tool that the user works with.
    """

    tech_category: EntityText = Field(
        description="Category: programming language, framework, database, etc.",
    )

Edge types use EdgeModel and carry their own attributes.

from zep_cloud.external_clients.ontology import EdgeModel

class WorksOn(EdgeModel):
    """The user is currently working on, building, or contributing to a project."""
    role: EntityText = Field(
        description="User's role: lead developer, contributor, maintainer, etc.",
    )

class UsesTechnology(EdgeModel):
    """The user actively uses or works with a specific technology."""
    proficiency: EntityText = Field(
        description="Proficiency level: beginner, intermediate, advanced, or expert.",
    )

Finally, wire these into the graph with source/target constraints using EntityEdgeSourceTarget, which defines which entity types can connect through which edge types:

from zep_cloud import EntityEdgeSourceTarget

client.graph.set_ontology(
    entities={"Project": Project, "Technology": Technology},
    edges={
        "WORKS_ON": (
            WorksOn,
            [EntityEdgeSourceTarget(source="User", target="Project")],
        ),
        "USES_TECHNOLOGY": (
            UsesTechnology,
            [EntityEdgeSourceTarget(source="User", target="Technology")],
        ),
    },
)

The code enforces that

• WORKS_ON can only connect a User to a Project

• USES_TECHNOLOGY can only connect a User to a Technology.

• Any relationship that doesn't match these constraints won't produce a typed edge.

Under the hood

When a conversation is ingested with a schema, Zep’s extraction pipeline runs through five steps:

1. Entity extraction identifies named entities in the text and classifies them against the defined types

2. Entity resolution merges duplicates (”Nexus” and “the Nexus project” become one node)

3. Fact extraction identifies relationships and outputs them as typed edges using the source/target constraints

4. Fact resolution detects contradictions and invalidates outdated facts, preserving history through Graphiti’s bi-temporal model, which tracks both event time and ingestion time

5. Temporal extraction parses time references and maps them to validity windows on each edge

The Pydantic schema defined above guides steps 1 and 3.

Entity types tell the extractor what to look for. Edge types with their constraints tell it what relationships to classify. Resolution and temporal processing happen automatically.

Every edge also carries explicit validity intervals (t_valid, t_invalid). When information changes (”Alex moved from Project Atlas to Project Nexus”), the old fact is invalidated, not deleted.

This way, you can query what’s true now as well as reconstruct what the state would have been at any point in time.

The output in practice

We ingest a conversation where a developer named Alex discusses their work (an active web app called Nexus, their tech stack, proficiency levels):

Querying for Project nodes returns Nexus with populated project_status and project_type attributes.

The node isn’t a generic “Topic” or “Object.” It’s a Project with structured fields as defined in the schema.

The edges are typed too.

• WORKS_ON carries role: lead developer

• USES_TECHNOLOGY carries proficiency: advanced for Python and Docker, proficiency: intermediate for TypeScript.

This can now filter projects by status, technologies by category, and query “which active projects use PostgreSQL” with a precise answer.

Context templates

The final piece is context templates, which assemble typed facts into a prompt-ready block.

You can define which edge types and entity types to include, and Zep formats them with temporal annotations into a single string injected into the agent’s prompt.

client.context.create_context_template(
    template_id="dev-context",
    template="""# PROJECTS
%{edges types=[WORKS_ON] limit=5}

# TECH STACK
%{edges types=[USES_TECHNOLOGY] limit=10}

# PROJECT DETAILS
%{entities types=[Project] limit=5}

# TECHNOLOGIES
%{entities types=[Technology] limit=10}""",
)

It looks like this:

Every entry in the resulting context block is typed, temporally annotated, and carries the attributes defined. Save the template once, reference it by ID in agent calls.

The 10/10/10 constraint and schema as a reasoning boundary

Zep enforces a hard limit of 10 custom entity types, 10 custom edge types, and 10 fields per type.

That’s intentional to force a dev to think about what matters in a domain rather than modeling everything.

The source/target constraints also act as guardrails on what an agent is allowed to remember. If a schema doesn’t include an edge type connecting Project to Competitor, the extraction model won’t create that relationship, even if a conversation mentions both.

The schema defines the space of valid memories.

This is the same principle behind typed function calling, where we constrain the LLM’s output space so that it can’t produce invalid arguments. Memory schemas apply that same constraint to what the agent stores.

Start with 3-4 entity types and 3-4 edge types that capture 80% of your domain logic, and add complexity incrementally.

Agent memory without schema discipline is a graph that behaves like a vector store.

In a way, you pay the cost of graph construction without getting the benefit of structured retrieval.

The schema is how you get that benefit back, and the fact that it’s Pydantic means there’s nothing new to learn.

You can find Zep’s GitHub repo here → (don’t forget to star 🌟)

👉 Over to you: have you defined a custom ontology for your agent’s memory, or are you letting the LLM decide the structure on its own?

Thanks for reading!

In 2023, AI products were thin wrappers. The LLM call was the entire workload, so cheaper models meant cheaper bills.

That’s not what AI queries look like today.

A single request can trigger a vector DB, web search, tools, and multiple LLM inferences before streaming a response back. The model got cheaper, but the infrastructure serving each query got heavier.

And when all of that runs inside a single deployable unit, auto-scaling makes it worse.

A spike in inference forces the entire app to scale, even when every other component is idle. The bill lands at 10x for a single-component problem.

AWS’ managed services decompose this by running each layer independently. Inference on Bedrock, vector search on Aurora pgvector, orchestration on EKS, each scaling on its own load curve.

Innovaccer runs this pattern for a healthcare AI platform serving 54M patients. After decomposing, they achieved 33% lower cloud costs and 65% less management overhead.

You can explore AWS’s resources for building AI-native SaaS here →

Thanks to AWS for partnering today!

Karpathy’s prediction about RL is coming true now!

Karpathy called reward functions unreliable and argued that a single reward number is too low-dimensional to teach an agent what “good” means for complex tasks.

To solve this, Agents need a knowledge-guided review as a higher-dimensional feedback channel.

Every major AI lab trains models with RL today (OpenAI, Anthropic, DeepSeek).

And their key bottleneck has always been the reward functions.

GRPO by DeepSeek worked well for math and code because the environment gave a binary signal.

But for real agent tasks, someone still has to hand-code the scoring function. That takes days and breaks every time the pipeline changes.

RULER (implemented in OpenPipe ART, 10k stars) addresses the exact problem Karpathy identified.

The reward criteria are defined in plain English, and an LLM evaluates each trajectory against that description to provide feedback for training.

We trained a Qwen3 1.4B agent that plays 2048 using GRPO with this exact workflow.

In this case, the agent saw the board, picked a direction, and RULER evaluated the outcome, all from this natural language definition.

You can see the full implementation on GitHub and try it yourself.

Here’s the ART Repo → (don’t forget to star it ⭐ )

ART GitHub Repo

Just like RLHF replaced manual rankings and GRPO replaced the critic model, natural language rewards are replacing hand-coded scoring functions.

RL reward engineering is now prompt engineering.

We also wrote a full walkthrough covering RL for LLM agents, from RLHF to GRPO to RULER. You can read it here →

Thanks for reading!

Knowledge graphs are insanely good at giving agents human-like memory!

Recently, we built an MCP-powered memory layer that can be shared across all your AI apps, like Cursor, Claude Desktop, etc.

Here’s a complete walk-through:

It’s built using a real-time knowledge graph.

Tech stack:

• ​Graphiti as a memory layer (open-source)

• Neo4j to store the knowledge graph

• Docker to self-host the MCP server

Everything is 100% open-source and self-hosted.

The anatomy of ~/.hermes folder

One folder controls everything your Hermes agent knows, remembers, and can do. Understanding its layout is the difference between treating Hermes as a black box and actually customizing it.

Here’s what lives inside and why each piece matters.

Configuration

config.yaml is the source of truth for everything non-secret, like model choice, terminal backend, tool enablement, and MCP servers.

The env holds your API keys and bot tokens and auth.json stores OAuth credentials.

Then there’s SOUL.md. It occupies slot #1 in the system prompt, before anything else loads.

It defines who the agent is, like its personality, tone, communication style, and hard limits. Everything the agent writes, creates, and remembers passes through this identity layer.

Knowledge

memories/ contains two tiny files. MEMORY.md (2,200 chars) holds project conventions, tool quirks, and lessons learned. USER.md (1,375 chars) holds your profile.

Both get injected into the system prompt as frozen snapshots at session start. When they fill up, the agent consolidates by merging entries, dropping redundancy, keeping only what’s dense and useful.

Capabilities

skills/ is where the learning loop lives. Each skill is a self-contained ability with a SKILL.md (the procedure), a references/ folder (docs the agent reads), and scripts/ (executable helpers).

Skills come from three sources. Either they are bundled with Hermes, downloaded from the hub via hub/, or created by the agent itself during your sessions.

Hermes ships with 687 skills across 18 categories, and you can add any GitHub repo as a custom tap.

Runtime state

sessions/ stores per-platform session metadata. state.db is the SQLite database with FTS5 indexing that backs tier 2 memory.

Automation

cron/ holds scheduled jobs in jobs.json and their outputs in output/. The gateway daemon ticks every 60 seconds and runs due jobs in isolated sessions. You describe schedules in plain English, Hermes converts them.

Extension + observability

plugins/, hooks/, and skins/ are the surface area for user customization. logs/ gives you agent.log, gateway.log, and errors.log for debugging.

You won’t manually edit most of these files. But knowing this layout means you understand exactly where identity, memory, skills, automation, and state live, and how they connect.

We wrote a full deep dive covering Hermes agent’s architecture, memory system, self-evolving skills, GEPA optimization, and setting up multiple specialized agents.

You can find the full deep dive here →

Thanks for reading!

P.S. For those wanting to develop “Industry ML” expertise:

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?

• Drive revenue?

• Can you scale ML models?

• Predict trends before they happen?

We have discussed several other topics (with implementations) that align with such topics.

Develop "Industry ML" Skills

Here are some of them:

• Learn everything about MCPs in this ​crash course with 9 parts →​

• Learn how to build Agentic systems in a crash course with 14 parts.

• Learn how to build real-world RAG apps and evaluate and scale them in this crash course.

• Learn sophisticated graph architectures and how to train them on graph data.

• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.

• Learn how to run large models on small devices using ​Quantization techniques.

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using ​Conformal Predictions.

• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.

• Learn how to scale and implement ML model training in this practical guide.

• Learn techniques to reliably test new models in production.

• Learn how to build privacy-first ML systems using ​Federated Learning.

• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.