Matthew Gallagher proved it by starting Medvi, an AI-run telehealth company, with $20,000 and no employees. It made $401 million in revenue in its first year.

Gallagher's stack was AI agents talking to each other. When two couldn't talk, he built more agents to handle the coordination. He tested the prompts himself and stepped in when something broke.

Alook (GitHub Repo) is an open-source, self-hosted platform that turns your coding agents into a real org chart.

Alook GitHub Repo

The way it works is that you create an agent, assign it a role, and give it a real email inbox.

Agents email each other and update you. They run as actual Claude Code or OpenCode sessions on your machine, with full access to your tools.

An org chart gives every agent a defined role and a reporting line. They coordinate with each other without you relaying a single message or fixing the wiring yourself.

Let’s set up Alook from scratch, build a four-agent org chart, and watch what happens once a real job gets handed to it.

Setup

Alook runs as a daemon on your own machine, and this one command connects it:

npx @alook/app onboard

It detects whatever coding agent runtime is already installed, Claude Code or OpenCode, and deploys the agent company.

This opens a local dashboard at http://localhost:15210.

From there, you start from a blank org chart or one of Alook’s pre-built templates, whichever’s closest to what you’re building.

Each agent on the chart is a real Claude Code or OpenCode session running on your machine, with full access to the same tools you use, and a real @alook.ai inbox.

The inbox is the coordination layer. Agents email each other, the same way a team would, instead of passing data through a trigger you wired by hand.

Building the company

Competitive intelligence usually means someone checking a pricing page, copying numbers into a spreadsheet, and doing it again tomorrow.

We’re going to replace that with four agents who build a price tracker, run it on a schedule, and email you the moment something changes.

First, we create the agents one by one, assigning them distinct roles and claiming their real @alook.ai inboxes:

• Atlas (CEO) is the single point of contact for the human. It delegates tasks to Mara.

• Mara (PM) turns Atlas's briefs into specs and routes them to Theo or Ren. She's the sole router on the chart.

• Theo (engineer) builds and maintains the scrapers for competitive intelligence.

• Ren (Ops and Customer-facing) notifies the human when a tracked change is detected.

Once the agents are live, we wire the reporting hierarchy: Atlas to Mara, and Mara to Theo and Ren.

Theo and Ren never talk to each other, or to Atlas directly. They only go through Mara.

This setup avoids building a chaotic AI group chat where every agent talks over each other and loses context.

Theo’s job touches the competitor’s site tracks, so he needs a way to scrape it reliably and schedule.

So we have given it access to the Bright Data CLI, which lets it scrape any website, provision custom scrapers if needed…all while avoiding IP blocks and CAPTCHAs that hit any agent scraping at real scale.

To build a custom scraper over any website, you can describe the page in plain English, and it builds the scraper that returns structured data:

Now let’s see how our AI company handles the real task.

Running the company

As mentioned above, we don't need to manage every agent ourselves. Instead, we just talk to Atlas, the CEO, and let the org chart handle the rest.

We ask him to track pricing on railway.app/pricing.

Atlas replies in the chat, and behind that reply, he's briefing Mara over email, and the thread shows up in the same window:

Mara turns the brief into a spec and hands it to Theo.

The spec covers a scraper for the page, timestamped snapshots, change detection, a daily run, and a report we can actually read.

Theo acknowledges the spec over email, the same way Mara and Atlas did, then builds it using the Bright Data CLI and reports back once it's running.

This is the scraper Theo just built, sitting in Bright Data's dashboard:

It's a real custom scraper that the agent provisioned itself by understanding the specified website, not a one-off CLI call that disappears after running. You can trigger it manually from this same screen, or call it straight using the API:

Mara relays the build-up to Atlas, and Atlas informs us in the same chat:

The company runs without you

Once Theo confirms the build is live, the job isn’t done. The schedule still needs to run, and someone still needs to watch what it finds.

The agent adds the scraper to the company calendar as a recurring 9am task, on its own.

This is Ren’s job. He watches the tracker’s output, and the moment the price on the page actually changes, he sends a note.

The whole loop runs unattended.

We handed Atlas one brief, and the org provisioned a scraper, scheduled it, and kept someone watching the output, with no further input from us.

Try it yourself

Every agent in this walkthrough ran as Claude Code, but Codex and OpenCode work the same way, as Alook lets you bring your own agent and gives whichever one you pick a role, an inbox, and a runtime that stays on.

Every completed task builds context for the next one, so the agents aren’t relearning the company from scratch on every run.

Every email between them gets logged the same way, so you can read back exactly how a decision got made.

Here’s the Alook’s Github repo →

And here’s the Bright Data CLI→

(don’t forget to star them 🌟)

Good day!

Teams usually validate their agents’ outputs by calling a frontier model as the judge. It works, but three problems stack up fast:

1. Cost: you’re hitting a frontier API on every turn, every tool call, every response. In production that burns millions.

2. Latency: bigger models, remote calls, slow reasoning on every check.

3. Blind spots: frontier models don’t actually know your domain. In finance, insurance, or healthcare, they miss the keywords and principles your work depends on.

In the video above, we walk through a different approach, which is to train your own small LLM judge.

Instead of a giant model, you start with a small one and let the system generate the training data for you. It decomposes your domain, samples synthetic examples, runs them through a debate arena where judges reach consensus, and then trains on the refined set.

This gives you a judge that’s cheaper, faster, and more accurate on your data than Gemini, Claude, or GPT, with an OpenAI-compatible endpoint you can even deploy on-prem.

We have shown the whole thing end-to-end, using a Claude Code plugin and a web interface, with a real insurance RAG grounding evaluator as the example.

You can get the plugin on GitHub here →

Here’s the full breakdown:

• 00:00 - Intro

• 00:12 - Three problems with using frontier LLMs as judges

• 01:05 - A different approach: train your own small judge

• 01:31 - How it works (synthetic data and a debate arena)

• 02:50 - Installing the Claude Code plugin

• 04:03 - Defining your task with /eval

• 04:34 - Example: an insurance RAG grounding evaluator

• 05:51 - Kicking it off and giving early feedback

• 06:26 - Choosing labels, domain, and strictness

• 08:30 - The web interface and dashboard

• 09:52 - Bringing your own example data (optional)

• 10:26 - The finished model: endpoint, accuracy, and speed

• 11:16 - Control, on-prem deployment, and interpretability

• 11:57 - Benchmarks vs frontier models and the GitHub repo

• 12:30 - Outro

Thanks to the PlurAI team for working with us on this today’s issue!

Hermes Mixture of Agents (MoA) explained

An agent runs on one model, and that model’s blind spots become the agent’s blind spots.

The same LLM does the reasoning, the tool calls, and the final answer, so wherever it’s weak, nothing else in the loop is there to catch it.

The usual workaround is to run the same prompt through a few models by hand and reconcile the answers. It works, but it lives outside the agent, so the tools, the memory, and the session are gone the moment that detour starts.

Hermes Agent by Nous Research shipped a mixture of Agents, which folds that whole process back inside the agent.

The unit you work with is a preset. Think of it as a recipe that names a few models to consult and one model to write the final answer, saved under a label you can remember and reuse.

So a preset might list GPT-5.5 and DeepSeek as the models to consult, with Opus as the one that replies. You set it up once, give it a name, and pick it later like any other model.

The models you consult run first and hand their analysis to the one writing the answer. That final model is the one that actually replies and makes the tool calls, now informed by several perspectives instead of one.

In Hermes, the preset you set up shows up as a model you can select.

So everything that already works in Hermes keeps working, like tool calls, follow-up iterations, memory, and the same session context behaves exactly as they do with a single model, because to the agent loop it is a single model.

The models can come from anywhere, i.e., a preset can mix OpenAI, Anthropic, DeepSeek, and Google, and it is not capped at two.

A few things follow from that design:

• It composes a model instead of choosing one. Several models covering each other’s blind spots can beat the strongest one on its own.

• It stays cheap to run. The models you consult see a stripped-down view of the conversation, so the extra calls stay light and the main context keeps its cache.

• It reaches past any single frontier model since it combines the providers already on hand.

• It is not a default setting so that it turns on only for the hard tasks where a second opinion matters, and stays off for routine work where speed wins.

The team behind Hermes has reported the effect on its own benchmark. A preset running Opus-4.8 over a GPT-5.5 reference scored higher than either model alone, by roughly six points and eight to eleven percent.

The lesson is not that one model has to win, but rather the fact that the best answer might not always come from a single model, and the agent should make blending them as easy as picking one.

If you want to learn how to set up Hermes, we wrote a full deep dive covering the Hermes agent’s architecture, memory system, self-evolving skills, GEPA optimization, and how to set up multiple specialized agents.

You can read it here →

Good day!

P.S. For those wanting to develop “Industry ML” expertise:

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?

• Drive revenue?

• Can you scale ML models?

• Predict trends before they happen?

We have discussed several other topics (with implementations) that align with such topics.

Develop "Industry ML" Skills

Here are some of them:

• Learn everything about MCPs in this ​crash course with 9 parts →​

• Learn how to build Agentic systems in a crash course with 14 parts.

• Learn how to build real-world RAG apps and evaluate and scale them in this crash course.

• Learn sophisticated graph architectures and how to train them on graph data.

• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.

• Learn how to run large models on small devices using ​Quantization techniques.

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using ​Conformal Predictions.

• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.

• Learn how to scale and implement ML model training in this practical guide.

• Learn techniques to reliably test new models in production.

• Learn how to build privacy-first ML systems using ​Federated Learning.

• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

It covers GRPO, the algorithm behind DeepSeek-R1, and a growing number of reasoning models. It picks up exactly where the RLHF chapter left off and asks, what if you do not need a learned reward model or a critic at all?

RL Nanodegree Part 10

You can read Part 10 of the course here →

It covers:

• Using verifiers instead of learned reward models

• Designing reward functions for verifiable tasks

• Why the critic becomes optional with cheap rewards

• The GRPO algorithm and group-relative advantages

• How PPO, DPO, and GRPO compare

• Trade-offs, including length bias and scope limits

• Hands-on GRPO training on math problems with Unsloth

Everything is covered from scratch, so no RL background is required.

You can read Part 10 of the course here →

RL Nanodegree Part 10

Why care?

DeepSeek-R1 demonstrated a tangible shift in post-training.

A model trained with GRPO and verifiable rewards developed strong reasoning capabilities, including chain-of-thought behavior that emerged purely from the RL process, without any supervised demonstrations of reasoning.

The four-model RLHF setup from the previous chapter (policy, reference, reward model, critic) collapsed to just two.

The memory cost of running four large models simultaneously was one of the biggest barriers to doing RLHF at scale. GRPO removed two of them.

It also matters conceptually. For any task where correctness is checkable (math, code, formal logic), a verifier gives an exact, unhackable signal.

The reward is grounded in fact, not in a model’s approximation of human judgment, leading to far better training.

This chapter covers the full picture, from the reward design to the GRPO objective, to a hands-on training run where you fine-tune a model to solve math problems using verifiable rewards.

Here’s what we have covered so far:

• Part 1: Foundations of Reinforcement Learning →

• Part 2: Markov Decision Processes and Value Functions →

• Part 3: Bellman Equations and Dynamic Programming →

• Part 4: Model-Free Learning →

• Part 5: Function Approximation →

• Part 6: Introduction to Deep RL and DQN →

• Part 7: Policy Gradients, REINFORCE, and Actor-Critic →

• Part 8: Proximal Policy Optimization in RL →

• Part 9: RLHF to Align Language Models with Human Feedback →

• Part 10: Verifiable rewards and GRPO →

Just like the MLOps course, each chapter will clearly explain necessary concepts, provide examples, diagrams, and implementations.

👉 Over to you: What topics would you like us to cover in this RL series?

Standard KG vs Zep’s temporal KG

When you employ a knowledge graph to serve as an agent’s memory layer, an LLM handles the extraction. It reads conversations and decides what entities to create, what relationships to track, and how to label everything.

Without any specific guidance, it labels everything generically.

• Project becomes a Topic.

• Database becomes an Object.

• The deployment tool becomes another Object.

• Every relationship between them gets the same RELATES_TO tag.

While the graph stores the right data, every node looks the same, and every edge means the same thing. So when the agent queries it, it starts looking like a similarity search across undifferentiated nodes.

Not only that, but facts change as well with time.

• A user could switch tech stacks.

• A customer could upgrade their plan.

• A project could go from active to completed.

The old facts don’t leave the graph but rather sit there with the same weight as the current one, and the naive retrieval has no way to tell which version is true right now.

So the agent serves stale context with full confidence.

Zep Graphiti (open-source with 28k+ stars) solves both problems at once:

• Start by defining the memory schema in Pydantic (entity types, edge types, attributes)

• The extraction model extracts and creates a graph against these specified types, so every node gets a specific type and every edge carries structured attributes.

• Contradiction detection invalidates outdated facts instead of keeping both

• Temporal annotations track when each fact was true and when it stopped being true.

Once this is ready, then at query time, retrieval filters by temporal validity before similarity search. Also, every fact that reaches the agent is typed, current, and structured.

The diagram below shows the full pipeline comparing a standard knowledge graph and a schema-guided knowledge graph side by side:

If you want to see this in practice, we covered it here with code →

And here’s the Zep Graphiti open-source GitHub repo →

(don’t forget to star it ⭐️)

Good day!

The core skills he listed were spec design, eval loops, and security oversight.

However, the tooling for this has been missing since practicing actual agentic engineering today still requires working across the editor, a terminal for scaffolding, a browser for testing, a cloud console for deployment, and a separate framework for evals.

The solution to production-grade Agentic Engineering is now actually implemented in Google’s Agents CLI. It covers the entire workflow in one place for scaffolding, evaluating, and deploying ADK agents.

Google's Agents CLI

It injects 7 skills into your coding agent that teach it ADK patterns, eval structures, and deployment targets.

After that, the coding agent automatically drives the entire lifecycle from natural language, and you don't need to leave your editor for any phase of the lifecycle.

Let’s walk through this end-to-end by building a RAG agent from scratch and deploying it as an internal knowledge assistant.

Step 1: Install Agents CLI

uvx google-agents-cli setup

This injects 7 bundled skills into your coding agent’s context, covering ADK code patterns, project scaffolding, evaluation setup with LLM-as-judge scoring, deployment configuration for Agent Runtime and Cloud Run, and Cloud Trace observability.

So each skill teaches the coding agent how a specific phase of the lifecycle works, so it can execute that phase directly from a natural language prompt.

One setup command installs these skills across every coding agent simultaneously. So Antigravity, Claude Code, Cursor, Codex, etc., all gain the same ADK expertise from a single install:

Step 2: Build the RAG Agent

Open the coding agent of your choice and describe the agent:

Build a RAG agent that ingests documents, retrieves relevant 
context, and answers questions with source citations. Use the 
ADK agentic_rag template with Gemini 3.5 Flash.

The coding agent activates its ADK skills and scaffolds the full project, as depicted below:

• Claude Code scaffolded the project from the ADK agentic_rag template with Vector Search as the datastore.

• It then identified that the template lacked citation support, so it rewrote the agent instruction to require grounded answers with inline citations and modified the retriever to surface source IDs with each document.

• It provisioned the datastore, ingested a synthetic Q&A corpus (12 entries on Python fundamentals), and ran a smoke test. The agent returned cited answers and correctly refused to hallucinate when the retrieval was down.

The injected skills know ADK patterns for retrieval-augmented agents, which is why the scaffold inherently included citation support and Vector Search config.

Step 3: Test locally

Next, we ask the coding agent to launch the ADK Web UI on localhost:

Spin up a local dev server so I can test this.

This launches an interactive chat interface where you can test the agent against real queries. Two things to verify here:

• First, does it retrieve and cite correctly? We ask “how to merge two dictionaries?” and the agent pulls the right context from the corpus, walks through both the merge operator and the update() method, and attaches [source: 1003] inline. Citations work.

• Second, does it handle missing context correctly? We ask “who won the FIFA World Cup in 2022?” which is a question the corpus has no answer to. The agent responds that it cannot answer based on the available documents.

Step 4: Evaluate before deploying

This is the most important step and the one that most agent tutorials skip entirely.

Generate 20 test scenarios for this RAG agent covering correct
retrieval, insufficient context where the agent should say it
doesn't know, multi-hop questions, and citation accuracy. Run
the full eval suite and show me the results.

The coding agent generated 20 test scenarios across four categories:

• 6 for correct retrieval (questions the corpus can answer)

• 5 for insufficient context (questions it should refuse to answer)

• 5 for multi-hop reasoning (questions requiring multiple documents)

• and 4 for citation accuracy.

Karpathy flagged this gap specifically and said 89% of teams running agents have observability set up, but only 52% have evals. Agents CLI lets you generate and run a full eval suite from a single prompt.

Results:

• Citation accuracy was perfect at 1.00 across all 20 cases. The agent never fabricated a source.

• But the hallucination score flagged an edge case, where, on questions outside the corpus, the agent sometimes appended general knowledge instead of saying it didn’t have enough context. The eval traced this to a single line in the instruction ("if you already know the answer to a simple question, you may respond directly without using the tools"), and removing that line from the instruction will solve this.

Step 5: Deploy to agent runtime

Deploy this to Agent Runtime in us-central1.

The coding agent first enhanced the project for Agent Runtime by adding the deployment entrypoint and infrastructure config.

It then deployed the agent to Google Cloud, and the whole process took about 2-3 mins.

Cloud Trace is enabled by default, so observability is built in from the first deployed request.

Step 6: Register to Gemini Enterprise

At this point, the agent is deployed and working, but it's only accessible to the developer who built it.

Anyone else who wants to use it needs the endpoint URL, the right API credentials, and enough context to know the agent exists in the first place.

In most teams, this is where useful agents quietly die. They work, but nobody outside the builder's immediate circle knows about them or can access them.

Asking the agent to do the following registers the app with the Gemini Enterprise platform, making it discoverable inside the Gemini Enterprise app across the entire org:

Register this agent to Gemini Enterprise.

Any team that has internal docs they want to make searchable can access the same knowledge assistant without setting up their own RAG pipeline. IAM controls who can access it, and the enterprise dashboard provides full observability.

This is what agentic engineering looks like with proper tooling, as Karpathy also described.

With one terminal session and six natural language prompts, and the agent went from an empty folder to a production assistant that the org can use.

You can find the Agents CLI on GitHub here →

Here’s the ADK documentation →

And here’s the Agent Platform →

👉 Over to you: what’s the biggest pain point in your current RAG setup that you wish you could automate away?

Thanks for reading, and to Google Cloud for partnering with us on today’s issue!

Pentesting firms don’t want you to see this.

An ​open-source AI agent​ just replicated their $50k service.

Here’s why this matters right now.

Teams are shipping faster than ever. AI writes the code, CI catches build failures, tests catch regressions, and observability catches outages.

But one more key question to ask is: What can an attacker do with this, right now?

It’s important to answer this question because several real-world examples make this hard to ignore:

• Moltbook exposed 1.5M auth tokens. The owner hadn’t written a single line of code.

• Tea App leaked 72,000 government IDs. The database was just open, no sophisticated hack needed.

• A researcher took control of a journalist’s computer through her own vibe-coded game, without a single click.

The code ran fine in all three cases, tests passed, and nothing raised a flag.

Because the bottleneck is no longer writing code, it’s understanding what that code actually exposes once it’s live. PR reviews miss auth edge cases, unit tests don’t probe broken access control, staging environments don’t simulate adversarial behavior, and business logic flaws look completely fine until someone decides to break them on purpose.

An automated approach is actually implemented in ​Strix, a recently trending open-source framework (26k+ stars) for AI pentesting agent.

It reviews any running app the way an attacker would:

• Crawls the app and maps every exposed route and flow

• Probes abuse paths dynamically, not just at build time

• Returns findings with proofs-of-concept and suggested fixes

It is benchmarked against 200 real companies and open-source repos, and it found 600+ verified vulnerabilities, including assigned CVEs.

It’s designed to fit into how modern teams already work: run it before a release, after major changes, or continuously as the app evolves.

​You can find the GitHub repo here →​ (don’t forget to star it)

Prompt engineering & loop engineering, clearly explained!

At its core, an agent is a while loop:

• The model runs

• It requests tool calls

• The tool results return to the context

• The model runs again until it stops requesting tools

This is not new. In fact, the ReAct pattern described this form of loop back in 2022-23, and almost every agent/framework runs a similar implementation of this (we implemented ReAct from scratch in pure Python here)

So loops aren’t a new thing at all, and the above implementation of loops was solved a long time ago.

What wasn’t solved is the loop around the above loop, and this is what Boris/Peter talked about recently.

In the most common setup, you are the loop around the loop.

• You write a prompt

• Read the turns the agent runs

• Write the next prompt

• And repeat, catching failures as it runs

Now there are attempts to let the system run the outer loop too, so that you can eliminate yourself.

• It starts on a schedule or an event

• It runs for many turns with no prompt in between

• It decides on its own when it’s done

• It comes back only when something needs you

Consider a failing test in CI to understand this.

In the current way, you paste the error into the agent, read the fix, run the tests, and paste the next failure back in until they pass.

So every turn goes through you.

The loop runs those same turns on its own.

It triggers on a schedule, reads the failure, drafts a fix on a branch, runs the tests, and feeds a failure back in as the next turn, until they pass or it hits a turn limit.

A separate reviewer checks the fix, opens a PR if it’s clean, and flags it for a human if it isn’t.

So the inner loop was always automatic. The part being automated now is your involvement in that loop.

None of that comes for free though, as expected.

→ Sitting in the outer loop gave you the flexibility to stop, possess project memory, and be the reviewer. But each of those now has to exist in the system.

→ Also, while sitting in the loop was slow, you understood the system.

But one big downside of taking yourself out is that you keep the ownership, but would likely lose the understanding of what’s being shipped.

→ Inherently, a loop doesn’t know when to stop on its own either. It will take the agent’s word that the work is done and stop on a fix while the tests still fail, so the stop has to be a real check, plus a turn or token cap to avoid infinite loops.

→ The context grows every turn, and the model gets worse as its context fills up.

So the loop should trim it and keep summaries instead of full history, move large outputs to files, and split subtasks into separate runs.

→ Lastly, the agent can’t be the one to check its own work, since it will pass whatever it wrote.

A separate model or a binary/deterministic test should provide that signal.

And the cost for all this adds up fast since every turn sends the whole context again, so a long loop can spend many times what a single prompt would.

If you want to dive deeper, we wrote a full breakdown, from the loop above to a run that finishes on its own, with the code behind each part.

Read it here →

​[Hands-on] Corrective RAG Agentic Workflow​

Corrective RAG (CRAG) is a common technique to improve RAG systems. It introduces a self-assessment step of the retrieved documents, which helps in retaining the relevance of generated responses.

Here’s an overview of how it works:

• First, search the docs with user query.

• Evaluate if the retrieved context is relevant using LLM.

• Only keep the relevant context.

• Do web search if needed.

• Aggregate the context & generate response.

The video at the top shows how it works!

Here’s our tech stack for this demo:

• Firecrawl for deep web search

• Milvus to self-host vectorDB.

• ​CometML’s Opik​ to trace and monitor

• LlamaIndex workflows for orchestration

Setup LLM

We will use gpt-oss as the LLM, locally served using Ollama.

Setup vectorDB

Our primary source of knowledge is the user documents that we index and store in a Milvus vectorDB collection.

This will be the first source that will be invoked to fetch context when the user inputs a query.

Set up web search tool

If the context obtained from the vector DB isn’t relevant, we resort to web search using Firecrawl.

More specifically, we use the latest v2 endpoint that provides 10x faster scraping, semantic crawling, News & image search, and more.

Tracing and Observability

LlamaIndex also offers a seamless integration with CometML’s Opik. You can use this to trace every LLM call, monitor, and evaluate your LLM application.

Create the workflow

Now that we have everything set up, it’s time to create the event-driven agentic workflow that orchestrates our application.

We pass in the LLM, vector index, and web search tool to initialise the workflow.

Kickoff the workflow

Finally, when we have everything ready, we kick off our workflow.

Check this out👇

In the video below, our workflow is able to answer a query that’s unrelated to the document. The evaluation step makes this possible:

If you want to dive into building LLM apps, our full RAG crash course discusses RAG from basics to beyond:

• ​RAG fundamentals​

• ​RAG evaluation​

• ​RAG optimization​

• ​Multimodal RAG​

• ​Graph RAG​

• ​Multivector retrieval using ColBERT​

• ​RAG over complex real-world docs ft. ColPali​

​The code for this issue is available here →​

👉 Over to you: What other RAG demos would you like to see?

Thanks for reading!

You can build it yourself, that’s not the question. It’s whether you want to spend tonight wiring up auth, a frontend, and a deploy pipeline before you watch the thing run.

Describe the app in the prompt box and Bolt.new builds the full stack in your browser: working UI, backend, and a live URL to share.

Prototype a dashboard for your latest analysis, stand up an internal tool, or pressure-test an idea before you commit real engineering hours. Then own what you ship, i.e., export the code, extend it, and take it anywhere. No lock-in.

The first 20 DailyDoseofDS readers get a free Bolt.new Pro code.

Grab yours right now here →

Thanks to Bolt for partnering today!

The AI engineering master stack for 2026!

We prepared this AI engineering master stack that covers ten layers from the model itself to running it safely in production:

Here’s what each layer comprises:

1) Foundations set how the model represents input through tokens, embeddings, transformer, attention, context window, positional encoding, and mixture of experts.

2) Model behavior covers how a trained model responds through pretraining, post-training, sampling, temperature, reasoning models, multimodality, and test-time compute.

3) Prompt engineering shapes the output through the prompt alone using system prompts, few-shot, chain-of-thought, structured outputs, prompt caching, self-consistency, and meta-prompting.

4) Retrieval feeds the model data it was never trained on through chunking, vector databases, hybrid search, reranking, retrieval eval, query rewriting, and GraphRAG.

5) Agents let the model take actions instead of only answering through function calling, ReAct, planning, reflection, multi-agent, computer use, and human-in-the-loop.

6) Context engineering controls what fills the context window across steps through context management, compaction, memory, MCP, agent harness, just-in-time retrieval, and structured note-taking.

7) Fine-tuning changes the weights when prompting and context fall short through SFT, LoRA, RLHF, DPO, distillation, GRPO, and RLVR.

8) Inference optimization makes the model cheap and fast to serve through quantization, KV cache, batching, speculative decoding, vLLM serving, FlashAttention, and PagedAttention.

9) Evaluation measures whether the system is actually correct through benchmarks, LLM-as-judge, golden datasets, hallucination detection, regression tests, trajectory evaluation, and red teaming.

10) LLMOps and safety keep the system reliable and safe in production through observability, cost tracking, guardrails, PII redaction, feedback loops, prompt injection defense, and model routing.

This grid above is the overview, but each layer is deep enough to be its own field with dedicated tooling.

We have covered every bit of it in the LLMOps course, starting from fundamentals to productions:

• Read Part 1 on fundamentals of LLMOps here →

• Read Part 2 on understanding the core building blocks of LLMs →

• Read Part 3 on the key components of LLMs, focusing on the attention mechanism, architectures like transformers and mixture-of-experts, and the fundamentals of pretraining and fine-tuning →

• Read Part 4 on decoding strategies, generation parameters, best practices, and the broader lifecycle of LLM-based applications →

• Read Part 5 on context + prompt engineering from a system perspective, in-context learning, types of prompts, and different prompting techniques →

• Read Part 6 on prompt versioning, defensive prompting, and techniques like verbalized sampling, role prompting, and more →

• Read Part 7 on context engineering, covering context types, context construction principles, and retrieval-centric techniques for building high-signal inputs →

• Read Part 8 on memory, dynamic, and temporal context in LLM systems, covering short and long-term memory, dynamic context injection, and common failure modes in agentic applications →

• Read Part 9 on evaluation methods and approaches for LLM-based applications, primarily focusing on building a strong understanding of the fundamental concepts →

• Read Part 10 on evaluation benchmarks in LLM applications, with task-specific methodologies, and the core tooling for evaluation of LLM apps →

• Read Part 11 on evaluation of multi-turn systems, tool use evaluations, tracing, and red teaming →

• Read Part 12 on LLM fine-tuning, parameter-efficient methods like LoRA and QLoRA, and alignment techniques such as RLHF, DPO, and GRPO →

• Read Part 13 on LLM inference optimization, KV caching, PagedAttention, FlashAttention, speculative decoding, and model parallelism →

• Read Part 14 on the fundamentals of LLM serving, including API-based access, inference with vLLM, and practical decisions.

👉 Over to you: What else would you add to the master tree?

6 components of context engineering

Here’s rough math on what determines your AI app’s output quality:

• Model selection: 15%

• Prompt: 10%

• Everything else (retrieval, memory, tools, query handling): 75%

We’ve seen teams obsessing over the wrong 25% when the actual problem lies elsewhere.

And this is exactly why “context engineering” has quietly become the most important skill in AI engineering today.

It’s the art of getting the right information to the model at the right time in the right format.

And it has 6 core components, as depicted in the visual below:

Prompting techniques

This is where most people stop. But even here, there’s more depth than people realize.

Classic prompting is about pattern recognition. You give the model examples, and it learns the format, style, and logic you want. Few-shot prompting still works remarkably well for structured tasks.

But advanced prompting is where things get interesting.

Techniques like Chain-of-thought prompting give the model thinking room. Instead of jumping straight to an answer, you ask it to reason step-by-step. This simple change can dramatically improve accuracy on complex problems.

Query augmentation

Users are lazy in writing queries.

When someone types “How do I make this work when my API call keeps failing?”, that’s almost useless to a retrieval system.

Query augmentation fixes this through several techniques:

• Query Rewriting: An LLM takes that vague question and transforms it.

• Query Expansion: Adding related terms and synonyms to cast a wider net.

• Query Decomposition: Breaking a complex question into sub-questions that can be answered independently.

• Query Agents: Using an agent to dynamically decide how to reformulate the query based on initial results.

Long-term memory

Say an agent has a great conversation with a user. The user shared preferences, context, and history. But as the session ends, it’s all gone.

Long-term memory fixes this with external storage:

• Vector Databases: Store embeddings of past interactions for semantic search.

• Graph Databases: Store conversations as relationships and entities.

The type of memory matters too:

• Episodic memory signifies specific events

• Semantic memory maintains general facts about the user, and

• Procedural memory handles how the user likes things done.

Open-source tools like ​Zep Graphiti​ make this accessible, and you don’t need to build from scratch.

Short-term memory

Short-term memory is simply the conversation history. This one seems obvious, but it’s often mismanaged.

And here’s where teams mess up:

• Stuffing too much into the context window (noise drowns out signal)

• Not including enough (model lacks critical information)

• Poor ordering (important context buried at the end)

• No summarization strategy for long conversations

Knowledge base retrieval

Most teams think about this as RAG, but that’s too narrow. RAG is one pattern, not the whole picture.

The real question is: How do you connect your AI to your organization’s data?

That knowledge lives everywhere, like: docs, wikis, databases, SaaS tools like Notion and Google Drive, APIs, and code repositories.

The retrieval pipeline has three layers:

• Pre-Retrieval: How do you chunk docs? What metadata do you preserve? How do you handle tables and structured data? How do you keep everything in sync?

• Retrieval: Which embedding model? Which retrieval strategy do you use: Vector search or hybrid with BM25? How do you re-rank?

• Augmentation: How do you format retrieved context, include citations, handle contradictions, etc?

Open-source tooling like ​Airweave​ solves this end-to-end. Instead of building custom connectors for every data source, you can sync your knowledge bases and get unified access to Notion, Google Drive, databases, and more.

You can get 10x improvements in retrieval quality without changing the model, but by just fixing the chunking strategy or properly syncing knowledge sources.

Tools and agents

A tool extends what the model can do because, without it, the model is stuck with just what’s in its weights and context window.

Moreover, an agent decides when and how to use those tools.

The basic loop looks like this: Query → Thought → Action → Observation → (repeat until goal satisfied) → Response

• Single-agent architecture works for straightforward tasks. Most chatbots and copilots fall into this category.

• A multi-agent architecture is better for complex workflows. You have specialized agents that collaborate. One does research, another writes, another critiques. They hand off work to each other.

​MCPs​ take this to the next step!

• Traditional tool integration requires N×M connections. If you have 3 models and 4 tools, you need 12 integration points.

• MCP changes this to N+M. Models and tools both connect to a standard protocol layer.

Some time back, prompt engineering made it sound like the magic was in crafting the perfect instruction.

Context engineering recognized that the real gains lie in the entire info pipeline instead:

• What context do you provide?

• Where does that context come from?

• How is it retrieved, filtered, and formatted?

• What can the model do with tools?

• What does it remember across sessions?

Over to you: How are you building your Agentic systems?

Good day!

Master.dev partnered with Anthropic and made their Claude Code course free for everyone. You can just dive in without any subscription or trial.

It’s taught by Lydia Hallie, who’s been an instructor with Master.dev for years and now works on the Claude Code team at Anthropic.

When she taught Claude Code live, it broke every platform record they have, with over 10,000 people tuning in.

Lydia has a knack for visualizing how tools work under the hood, which is exactly the mental model you need to stop guessing with AI and start directing it.

Learn Claude Code for free →

Thanks to Master.dev for partnering today!

Speculation is all you need!

Speculative decoding speeds up LLM inference, but the gain usually stalls around 2-3x. The cap comes from the drafter, the small model that proposes tokens for the large target model to verify, because it still generates one token at a time.

Modal recently released a new set of DFlash draft models for several Qwen models that move past this cap.

Running Qwen 3.5 122B-A10B with one of them reached over 1000 tokens/sec, up from 250 without speculation, and the output stays identical to normal decoding.

Let’s break down how speculative decoding works, why acceptance length decides the speedup, and what these new drafters changed to push it higher.

How speculative decoding works

A small draft model generates the next several tokens, and the large target model verifies all of them at once in a single forward pass.

Every token the target agrees with is kept. At the first mismatch, the target replaces that token with its own prediction, discards everything after it, and generation continues from the last accepted token.

The kept tokens are exactly what the target would have produced on its own, so the output matches standard decoding with no quality loss.

Why the drafter caps the speedup

A standard drafter is autoregressive, generating one token at a time, so the drafting step becomes the bottleneck and holds real speedups near 2-3x.

DFlash replaces that autoregressive drafter with a block diffusion model that denoises a full block of tokens in one parallel pass rather than left to right.

The drafting cost then stays flat, no matter how many tokens it proposes ahead.

The drafter also does not work from the token sequence alone. As the target model reads the context, each of its layers produces hidden states, and DFlash pulls those representations from several layers and feeds them into the draft model.

So the drafter starts from the target’s own internal view of the context instead of guessing from raw tokens, which makes its proposed tokens line up with the target more often.

What decides the speedup?

How often those proposals get accepted is captured by the acceptance length, the count of drafted tokens the target accepts per forward pass.

Acceptance length maps almost linearly to the speedup, because decode is memory-bound rather than compute-bound.

Each step spends most of its time moving model weights from GPU memory into the compute units, not on the matmuls, and a model above 100B parameters means reading tens of gigabytes every step.

Reading those weights takes the same time whether the pass checks one token or a full block. So every token accepted in a pass is nearly free throughput, while every rejected token still costs time to draft.

This is also why drafting a thousand tokens at once does not help. The target keeps the run only up to its first mismatch, so a 1000-token block that misses on the 5th token throws away the other 995, along with the compute spent drafting them.

Draft length must be balanced:

• Too few draft tokens per pass barely move throughput

• And too many draft tokens likely waste compute on tokens discarded past the first miss.

This is why real production gains not only depend on how tokens get drafted, but also on how many survive verification.

Modal’s new strategy for DFlash

Modal recently released a new set of DFlash draft models for several Qwen models on HuggingFace.

They trained the drafters on the target’s own outputs and on real production traffic, so they propose tokens the target actually emits.

The run then survives longer before the first miss, which pushed acceptance length from a baseline of 3 to over 9.

They benchmarked Qwen 3.5 27B on one B200.

An acceptance length of:

• 2 gave 1.86x speedup.

• 4 gave 3.57x speedup.

• 8 gave 5.62x speedup.

Running Qwen 3.5 122B-A10B with one of these tuned drafters reached over 1000 tokens/sec at concurrency 1 on a B200, against 250 without speculation.

If you are running speculative decoding in production, the quality of the drafter will always decide how high the acceptance length can reach.

Training the drafter on the workload is a practical way to get there. The DFlash drafters are already integrated with vLLM, SGLang, and Transformers, with draft models on HuggingFace for Qwen and several other model families.

DFlash draft models on HuggingFace →

​Our full breakdown on speculative decoding covering drafting, verification, rejection sampling, and several other techniques to optimize it →​

👉 Over to you: If you’re self-hosting inference, would you train a custom drafter on your own production traffic, or run a generic one off the shelf?

Build Agents that can learn like humans​

Reinforcement learning for LLMs always had one major problem: manually defining reward functions.

You have to figure out how to score model outputs, handle edge cases, and tune everything until training works.

​ART (Agent Reinforcement Trainer)​ is an open-source framework that solves this with a simple approach:

OpenPipe ART GitHub repo

• Let the agent attempt tasks multiple times

• An LLM judge relatively grades each attempt

• The model learns from what worked vs what didn’t

Notice that it needs no manual reward engineering. You don’t have to manually score output or tune penalties. You just need to know which attempt was better, and LLM judges are naturally great at that comparison.

If this sounds familiar, it’s because it’s the core idea behind GRPO (Group Relative Policy Optimization), the algorithm that made DeepSeek R1 so effective.

But here’s what makes ART different.

Most RL frameworks are built for simple chatbot interactions.

One input, one output, and done. But real-world agents search through documents, invoke APIs, and reason across multiple steps before completing a task.

Most RL frameworks can’t handle this complexity, but ART can.

It provides:

• Native support for tool calls and multi-turn conversations.

• Integrations with LangGraph, CrewAI, and ADK.

• Efficient GPU utilization during training.

It handles the entire infrastructure using vLLM to serve models and generate trajectories, and Unsloth to apply GRPO and run training.

These are two of the most popular open-source LLM frameworks, stitched together into one seamless workflow.

This means you can easily fine-tune a small open-source model to outperform sophisticated closed-source models on your specific task.

In the video below, we walk through the entire process:

• setting up ART,

• understanding how the GRPO training loop works,

• and fine-tuning Qwen to intelligently search through emails.

Everything runs locally. 100% open-source.

What you learn here can be applied to any problem.

​You can find the ART GitHub repo here →​ (don’t forget to star it)

It has the notebook from the video, plus additional examples!

Thanks for watching!

Doc Holiday solves one of engineering’s most persistent problems: knowledge degeneration.

Every time code ships, the gap between what the product does and what the rest of the company knows widens. Documentation is usually the first thing to fall behind. The support team spends the next sprint answering questions that the last release already answered.

Drop it into your CI/CD pipeline and connect any upstream source, like Notion, Jira, Slack, Zendesk, Confluence, or Google Docs.

When a PR merges, it reads the commit history, linked tickets, and connected specs, then generates the changelog, release notes, and documentation updates automatically.

It keeps documentation in sync with the codebase without adding a step to the release process.

Get started with Doc Holiday here→

Thanks to Doc Holiday for partnering today!

Loop engineering, clearly explained!

Every agent, underneath whatever framework you’re using, runs the same loop.

while True:
    response = model(context)
    if response.has_tool_calls():
        context += run_tools(response.tool_calls)
    else:
        break

• Send the context to the model

• It responds with tool calls

• Run those tools

• Append the results to the context

• And send it back.

It keeps going until the model replies without asking for a tool.

That loop is short, and it’s nearly identical across LangGraph, the OpenAI Agents SDK, and Claude Code, so nobody competes on the while statement.

This is exactly why the engineering effort moved somewhere else.

More specifically, the model and the loop are the parts you don’t write.

What you write is everything around it, like when the loop stops, what stays in the context, which tools the model can reach, and how you check the result.

So let’s go through the loop itself, then the four parts of it that are hard to get right.

Where the engineering actually moved

It moved outward, into the layers that wrap the model.

• Prompt engineering is the words you send.

• Context engineering is everything the model sees on a turn, not just your instructions.

• Harness engineering is the code around the model that runs tools, tracks state, and recovers from errors.

• Loop engineering is the outer cycle that decides what the agent works on and when it’s done.

Each layer wraps the one before it, so your prompt is now one input to a much larger system.

Here’s how these systems break today.

1) Ending a turn is not finishing the job

The loop stops on exactly one condition, when the model replies without a tool call. So it ends the moment the model decides it’s finished, which means the model is judging its own completion.

while True:
    response = model(context)
    if response.has_tool_calls():
        context += run_tools(response.tool_calls)
    else:
        break

That judgment is often wrong. A coding agent makes an edit, returns a confident summary with no further tool call, and the loop exits even though it never ran the tests, or ran them and they failed. The turn ended, but the task wasn’t done.

Since you can’t trust the model’s own stop signal, you add conditions it doesn’t control:

Max iterations, a hard cap so a stuck agent can’t run forever.

• Budget and time limits, a ceiling on tokens, money, and wall-clock seconds.

• No-progress detection, which catches the agent repeating the same call with the same arguments.

• A real completion check, an automated condition that proves the job is done.

The completion check is super important, because it’s the only brake that replaces the model’s self-assessment with an objective signal.

“Done” should mean the tests pass, not the model reporting that it’s done. Claude Code’s /goal command works this way, running the loop until a verifiable condition holds and using a separate model to confirm it.

2) Context rot and the doom loop

The longer a loop runs, the more its context fills with junk, like old tool outputs, abandoned dead ends, and stale reasoning. Model quality drops as that pile grows, which the field calls context rot.

The loop turns rot into a spiral, where a rotted context produces a worse decision, which adds more noise, which rots the context further.

The community calls this the doom loop, and the agent gets less useful the longer it runs. LangChain added middleware specifically to detect doom loops in their harness.

You solve this by treating context as a budget, not a bucket:

• Compaction, summarizing the conversation once it gets long, and continuing from the summary.

• Offloading, pushing large outputs to a file, and keeping only the slice you need.

• Sub-agents, handing a messy subtask to a separate agent so that only its clean result returns.

The instinct is to keep everything in case it matters later. The skill is knowing what to throw away.

3) Tool design changes inside a loop

Adding tools makes selection harder, not easier.

Give the agent a hundred overlapping tools and it loses track of which one to call, so a small set of focused, non-overlapping tools works better.

Anthropic’s rule of thumb is that if a human engineer can’t say for certain which tool fits, neither can the agent.

Vercel found that cutting an agent’s available tools raised its success rate.

Two more properties matter specifically because this is a loop, not a single call:

• Writes have to be safe to repeat, since the loop retries a failed step, and a retried “create customer” that makes a second customer leaves you with duplicate records and double billing.

• Error messages have to tell the agent what to do next, not just what went wrong, because in a loop, that message becomes the input to the next turn, so a vague error wastes a turn and a precise one fixes the bug.

4) Put something in the loop that can say no

The completion check from earlier is one case of a wider rule.

Whatever decides if the work is good can’t be the same model that produced it. A model asked to grade its own output will usually pass it, so a loop with no outside check is just an agent agreeing with itself.

So you separate the maker from the checker. One agent writes the code, and a separate signal grades it, either something hard like a failing test or type error, or a second model running with different instructions.

That check is what lets you actually leave the loop alone, because now something other than the author decides when it’s right.

What is user’s job now?

Prompting steers the agent move by move.

Loop engineering involves building the system that steers it, and then stepping back. The work becomes three artifacts:

• The goal is written as a success criterion that the agent can check itself against.

• The loop, with brakes, so it stops for the right reasons.

• The verifier, so “done” is proven, not claimed.

Karpathy said something along these lines too.

His AutoResearch project runs exactly this, an agent that tweaks a training script, measures the result, keeps what works, and discards what doesn’t, with no human editing the code between rounds. He arranges it once and lets it run.

You don’t need an overnight autonomous agent on day one. Build up to it:

1. Start with the basic loop and add a max-iteration cap, a timeout, and a cost ceiling immediately.

2. Define “done” as an automated check before you start, not a vibe afterward.

3. Protect the context, compacting long runs, offloading big outputs, and isolating messy subtasks.

4. Audit your tools, keeping them few and focused, making writes safe to repeat, and rewriting errors so an agent can act on them.

5. Put a critic in the loop, and only go fully hands-off once you trust the thing that says no.

The takeaway

Loop engineering isn’t a framework you install but rather a shift in where you spend effort.

The model is becoming a commodity, and the loop around it is where the engineering now lives.

The builders getting value this year stopped asking what to tell the agent and started asking what system would do the work without them.

👉 Over to you: what’s the first brake you’d add to a loop you already run, a completion check, a budget cap, or a separate verifier?

​To dive deeper into harness engineering, we covered it in detail here →

Thanks for reading!

P.S. For those wanting to develop “Industry ML” expertise:

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?

• Drive revenue?

• Can you scale ML models?

• Predict trends before they happen?

We have discussed several other topics (with implementations) that align with such topics.

Develop "Industry ML" Skills

Here are some of them:

• Learn everything about MCPs in this ​crash course with 9 parts →​

• Learn how to build Agentic systems in a crash course with 14 parts.

• Learn how to build real-world RAG apps and evaluate and scale them in this crash course.

• Learn sophisticated graph architectures and how to train them on graph data.

• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.

• Learn how to run large models on small devices using ​Quantization techniques.

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using ​Conformal Predictions.

• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.

• Learn how to scale and implement ML model training in this practical guide.

• Learn techniques to reliably test new models in production.

• Learn how to build privacy-first ML systems using ​Federated Learning.

• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

It covers RHLF, and this is the chapter where everything we have built across the series (value functions, policy gradients, actor-critic, PPO) converges into the actual pipeline used to align language models like ChatGPT, Claude, and Gemini.

Reinforcement Learning Nanodegree Part 9

You can read Part 9 of the course here →

It covers:

• Limitations of instruction fine-tuning

• Turning human comparisons into reward

• Training the reward model

• The four-model RLHF setup

• Keeping the model from drifting

• Reward hacking and over-optimization

• Hands-on reward model training

• DPO as a simpler alternative

• When verifiable rewards work better

Everything is covered from scratch, so no RL background is required.

​You can read Part 9 of the course here →

Reinforcement Learning Nanodegree Part 8

Why care?

RLHF is one key reason why modern language models feel different from the GPT-2 era.

Pre-training gives a model knowledge. Supervised fine-tuning teaches it the format of a conversation.

But the step that turns a text-completion engine into something that follows instructions, avoids harmful outputs, and responds in a way that feels genuinely useful is RLHF.

Every major model release over the past three years has included some version of this pipeline.

Understanding RLHF also makes the ongoing alignment conversation far more concrete and helps understand reward hacking, over-optimization, sycophancy, length bias.

This series has been building toward this chapter from the start.

Here’s what we have covered so far:

• Part 1: Foundations of Reinforcement Learning →

• Part 2: Markov Decision Processes and Value Functions →

• Part 3: Bellman Equations and Dynamic Programming →

• Part 4: Model-Free Learning →

• Part 5: Function Approximation →

• Part 6: Introduction to Deep RL and DQN →

• Part 7: Policy Gradients, REINFORCE, and Actor-Critic →

• Part 8: Proximal Policy Optimization in RL →

• Part 8: RLHF to Align Language Models with Human Feedback →

Just like the MLOps course, each chapter will clearly explain necessary concepts, provide examples, diagrams, and implementations.

👉 Over to you: What topics would you like us to cover in this RL series?

Good day!

Most SDKs give you tools to build an agent. Strands gives you tools to build an agent harness so you can control what it does, when it does it, and how far it goes.

Build an agent harness. Control it end-to-end.

Get Started →

Thanks to AWS for partnering today!

Build your own AI avatar with human-like memory

NaiveRAG is fast but dumb.

GraphRAG is smart but slow.

This open-source solution fixes both.

RAG systems have a fundamental problem: They treat documents as isolated chunks. No connections. No context. No understanding of how things relate.

Graph RAG addresses this, but traditional graph databases become painfully slow for real-time applications.

What if you could combine the speed of vector search with the intelligence of knowledge graphs?

That’s exactly what we have built and shared in the video above.

A real-time AI Avatar that uses a knowledge graph as its memory. You talk to it directly, and everything happens in real-time.

Watch the video for the full demo and code walkthrough. We’ve open-sourced everything.

To power this, we used Zep’s knowledge retrieval system (open-source).

What makes it fast:

↳ Smart retrieval algorithms that avoid full graph search.

↳ Fine-tuned Qwen3 and Gemma models with <10ms embedding and <50ms reranking.

↳ S3 + hot caching for dense vector and BM25 search instead of traditional vector databases.

Zep’s open-source framework Graphiti is available under Apache 2.0, so you can easily self-host it.

You can find Zep’s GitHub repo here →

You can find the code here →

​9 MCP projects for AI Engineers​

We have covered several MCP projects in this newsletter so far.

Here’s a recap along with visuals & full code walk-through issues:

#1) 100% local MCP client

• An MCP client is a component in an AI app (like Cursor) that establishes connections to external tools. Learn how to build it 100% locally.

• ​Full walkthrough →​

#2) MCP-powered Agentic RAG

• Learn how to create an MCP-powered Agentic RAG that searches a vector database and falls back to web search if needed.

• ​Full walkthrough →​

#3) MCP-powered financial analyst

• Build an MCP-powered AI agent that fetches, analyzes & generates insights on stock market trends, right from Cursor or Claude Desktop.

• ​Full walkthrough →​

#4) MCP-powered Voice Agent

• This project teaches you how to build an MCP-driven voice Agent that queries a database and falls back to web search if needed.

• ​Full walkthrough →​

#5) A unified MCP server

• This project builds an MCP server to query and chat with over 200+ data sources using natural language through a unified interface powered by MindsDB and Cursor IDE.

• ​Full walkthrough →​

#6) MCP-powered shared memory for Claude Desktop and Cursor

• Devs use Claude Desktop and Cursor independently with no context sharing. Learn how to add a common memory layer to cross-operate without losing context.

• ​Full walkthrough →​

#7) MCP-powered RAG over complex docs

• Learn how to use MCP to power an RAG app over complex documents with tables, charts, images, complex layouts, and whatnot.

• ​Full walkthrough →​

#8) MCP-powered synthetic data generator

• Learn how to build an MCP server that can generate any type of synthetic dataset. It uses Cursor as the MCP host and SDV to generate realistic tabular synthetic data.

• ​Full walkthrough →​

#9) MCP-powered deep researcher

• ChatGPT has a deep research feature. It helps you get detailed insights on any topic. Learn how you can build a 100% local alternative to it.

• ​Full walkthrough →​

👉 Over to you: What other MCP projects would you like to learn about?

Good day!

Claude Code has two built-in tools for the web, web_fetch and web_search, and neither is meant for true scraping:

• web_fetch runs the page through a smaller model and returns a summary capped at a 125-character quote, so you can’t pull full specs or thread content. web_search returns ranked results, not the page itself.

• curl returns raw HTML but gets blocked on Amazon, LinkedIn, and most protected sites, and it can’t render JavaScript pages.

The Bright Data CLI has already been a reliable solution to handle this limitation.

It provides a scraping infrastructure that handles bot detection, CAPTCHAs, and browser rendering.

It also has pre-built extractors for 40+ platforms (Amazon, LinkedIn, Instagram, TikTok, YouTube, Reddit, etc.), but for all other sites, you need to rely on standard markdown scraping. It returns the page as markdown that you parse field by field and rewrite whenever the site’s layout changes.

Scraper Studio is a new addition to the CLI now, where you give the CLI a site and the fields you want, and it generates a custom scraper, publishes it as an API, and repairs it when the page changes…all with natural language.

Let’s build a custom scraper ourselves.

Bright Data setup

Bright Data skills and CLI (open-source) add scraping infrastructure that handles everything web_fetch and curl can’t.

Start by installing the Bright Data skills and CLI:

npx skills add brightdata/skills

curl -fsSL https://cli.brightdata.com/install.sh | sh

This installs several skills covering scraping, search, structured data feeds, MCP orchestration, SDK best practices, and the bdata CLI.

Building a custom scraper

This is how we prompt Claude Code:

Create a custom scraper using Bright Data scraper studio
for this page: https://blog.dailydoseofds.com/p/the-production-harness-for-ai-built.
Figure out the relevant fields you can extract
from this to build the custom scraper. Tell me the
API endpoint once done, name, wrap up it up in a python script
that can directly invokes the API endpoint.

The scraper-builder skill loads and first checks Bright Data's supported domains. Substack has no pre-built scraper, so this is a custom-scraper case:

Claude Code then fetches the page and reads its structure to decide what to extract.

It then builds the collector with one CLI command:

brightdata scraper create 
 "https://blog.dailydoseofds.com/p/the-production-harness-for-ai-built"

The CLI creates a template, triggers the AI generation, and returns a collector. To verify it, Claude Code runs the collector against the same URL:

Finally, as instructed, Claude Code wraps those into a Python script that triggers the collector, polls for the response, and writes the records to JSON:

TRIGGER_URL = "https://api.brightdata.com/dca/trigger?collector=...&queue_next=1"
DATASET_URL = "https://api.brightdata.com/dca/dataset"
HEADERS = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}

def trigger(url):
    resp = requests.post(TRIGGER_URL, headers=HEADERS, json=[{"url": url}], timeout=30)
    resp.raise_for_status()
    return resp.json()["response_id"]

def fetch_results(response_id):
    while True:
        resp = requests.get(DATASET_URL, headers=HEADERS,
                            params={"id": response_id}, timeout=30)
        data = resp.json()
        if isinstance(data, list) and data:
            return data
        time.sleep(3)

From a one-line prompt, you end up with a named collector, a callable API, and a script that hits it, for a site that had no pre-built scraper.

So essentially, instead of a fixed catalog of scrapers, you can provision a new one for whatever site your project needs, generated from inside the agent that’s building the project.

The data source becomes something you spin up as you go, shaped to the exact fields you want. For anyone building against the open web, that is a strong thing to have one command away.

Note: None of this is locked to the terminal. Generating the collector, approving the fields, and publishing the API all run directly in the Scraper Studio dashboard, and that’s also where you repair a scraper in plain language when a page change turns a field into undefined.

You can try it here →

Pre-built extractors cover 40+ popular platforms. For everything else, Scraper Studio turns a plain-language description into a custom scraper, a callable API, and a collector you own, whether you build it from the CLI or the dashboard.

Why Bright Data?​

Building AI apps capable of interacting with real-time web data can feel impossible. Here are the challenges:

• You must simulate human-like interactions.

• You must overcome site blocks and captchas.

• You must scrape accurate and clean data at scale.

• You must ensure compliance with all legal standards.

​Bright Data provides the complete infrastructure to handle data extraction, user simulation, and real-time interactions for your AI apps across the web.

With ​Bright Data, you can:

• Access clean data from any public website with ease.

• Simulate user behaviors at scale using advanced browser-based tools.

• Enable AI models to retrieve real-time insights with a seamless Search API.

Resources to go deeper:

• Bright Data Scraper Studio →

• The Bright Data CLI repo →

👉 Over to you: Which custom scraper will you build today?

Thanks to ​Bright Data for letting us use their scrapping infra and partnering with us on this demo.

Visual guide to Bi-encoders, Cross-encoders & ColBERT​

So many real-world NLP systems, implicitly or explicitly, rely on pairwise sentence (or context) scoring in one form or another.

• ​RAG systems​

• QA systems

• Duplicate text detection systems, etc.

The visual depicts three popular approaches used in the industry to handle this:

Let’s understand them one by one!

We covered them with implementation here:

1) ​Bi-encoders and Cross-encoders for sentence pair similarity scoring​.

2) ​AugSBERT for sentence pair similarity scoring​.

3) ​A deep dive into ColBERT and ColBERTv2 for improving RAG systems (with implementation).​

1) Cross-encoders

These are conceptually one of the most powerful approaches.

• Concatenate the query text and the document text.

• Encode it using a BERT-like encoder model.

• Apply a transformation (a dense layer) to the [CLS] token representations to get a similarity score.

Since the model attends to both contexts, this produces an incredibly semantically expressive representation.

But it does not scale because if you have 1B documents, you must do 1B forward passes to determine the most relevant documents to a query.

2) Bi-encoders

• Encode the query and the documents separately.

• Compute the cosine similarity between the [CLS] token of the query and the document.

This is highly scalable since the document embeddings can be computed offline.

But we lose all the interaction and simply “hope” that the entire information about the query and the document is well summarized in the [CLS] token.

3) ColBERT

This brings together the power of cross-encoders and the scalability of bi-encoders.

Encode the query and the documents separately.

• Compute a late interaction matrix, which contains similarity scores (dot product) between all query tokens and all document tokens.

• For every token, determine the max score across all document tokens.

• Sum these max scores to get a matching score.

Advantages:

• Like bi-encoders, it is highly scalable since document embeddings can be computed offline.

• Like cross-encoders, it maintains cross-interactions between the query and the document tokens (called late interaction).

We covered them with implementation here:

• ​Bi-encoders and Cross-encoders for Sentence Pair Similarity Scoring.

• ​AugSBERT for Sentence Pair Similarity Scoring.

• ​A deep dive into ColBERT and ColBERTv2 for improving RAG systems (with implementation).​

Over to you: What are some other advantages of ColBERT?

Thanks for reading!

When an AI agent deletes a production database, the model is almost never the weak link.

This sounds counterintuitive, but the dangerous part is rarely the code the agent writes.

Instead, it is what sits around that code, the access it inherits, and the boundary nobody put in front of it.

A model will happily generate a button that issues account credits and never question who is allowed to click it, because authorization was never in the prompt.

Today, let’s look at this in action and the solution to avoid this in production!

Our test run

We asked Fable 5 to build a small internal support console with one button that issues an account credit.

It worked on the first run, and on our laptop, the credit was applied to the balance the instant we clicked:

Issuing a credit is an UPDATE against production rows, and it commits real money permanently.

In this case, the agent generated the code end-to-end and stopped. Any caller could reach it, and no role decided whether they were allowed, and nothing was recorded of what they did.

Of course, it built exactly what we asked for. While the model knows well that a button moving money in production needs an access check, it just didn’t add one because the prompt didn’t ask it to.

Why does this happen?

In one widely-reported case, eleven ALL-CAPS warnings not to touch production did not stop an agent from deleting the database.

Instructions live inside the same context that the model is free to override, so they are guidance, not enforcement.

Ideally, these controls should live in the platform the app runs on, not in the app.

Because the moment each app handles its own permissions, you get a hundred tools with a hundred different rules and no central way to manage them. That sprawl is what Shadow AI really means.

The boundary should belong to the runtime where:

• Credentials are scoped server-side

• Access runs through shared permission groups

• And writes are gated and logged centrally.

An agent can’t emit that, because it lives outside the app it builds.

Solution

A smart approach to this is now actually implemented in Retool.

Essentially, instead of asking the model to add the boundary it was never going to add, you can keep building wherever you are fastest (Claude Code, Cursor, Codex, etc) and let the Retool runtime supply the boundary on the way to production.

To demonstrate this, we took the same app from earlier and dropped its React bundle into Retool:

Once done, it parsed the components, pulled them in, and walked us through pointing the app’s data calls at a Retool resource, the governed connection that fronts the real database, everything automatically.

We wrote none of the access control ourselves.

Note: Setup wise, we only signed up for Retool to get production infrastructure for the app. The React code was generated on its own, with no knowledge that Retool would later run it, which is the most important point here.

Nothing in the build was written for Retool, and it still imported cleanly.

That said, there’s a Retool MCP server if you’d rather have the runtime guide the build from inside your coding agent, but we didn’t use it here. The import worked on a plain, Retool-unaware bundle.

The moment those data calls ran through the resource, the app inherited identity, permissions, and an audit trail from the runtime.

As depicted in the video above, the credit call was now routed through the resource layer.

Moreover, SSO resolved the caller to a real identity. And permission groups decided whether their role can issue credit.

The mutating query trips an approval gate that is on by default, so the write waits for a human instead of committing silently. Each run lands in an audit log with a name and a timestamp.

Nothing in the app changed. It’s the same credit button with the same function behind it. What changed is that the write now runs through Retool’s boundary.

The model makes writing the code almost free.

It does almost nothing for the identity, permissions, and audit trail that make that code safe to run in production, because those come from where the app runs, not from the model.

With Retool, you can build the app anywhere you like and push it to Retool’s runtime to make it safe to ship. None of this slows down the building process.

You can try it out yourself here →

Thanks to Retool for partnering with us today!

HarnessX: A harness that compiles itself​

Every harness improvement so far has come from a human editing code by hand.

Anthropic strips planning steps out of Claude Code when a stronger model ships. Manus rebuilt its agent five times in six months, removing complexity each round.

The entire engineering runs on human judgment about what to change and when. HarnessX is a way that allows the system to make those edits itself:

The trick is to treat the harness as a first-class object, the way we already treat model weights.

Once it’s a typed, editable artifact, it can be optimized from its own execution traces.

And evolving a harness maps cleanly onto ​reinforcement learning​.

• The harness is the state.

• An edit is the action.

• The trace plus a score is the feedback.

• A new version is the update.

Once you see it that way, the failure modes also become easily evident, like reward hacking, catastrophic forgetting, and under-exploration.

And they are the same problems that break model training, showing up the moment a system starts editing its own scaffolding.

But the good thing is that because the failure modes are known ahead of time, the defenses are part of the design rather than patches added later.

Every edit goes through a loop before it is applied:

• One stage compresses the execution traces into a specific failure.

• The next plans a single change.

• The third writes the edit.

• The fourth runs it and checks the result.

A deterministic gate decides what should persist, so a new version replaces the current one only if it scores higher on the evaluation set; otherwise, it is dropped and the loop reruns.

This self-editing process only works because of how the harness is built. It is composed of typed processors that attach at fixed points in the execution lifecycle, so the system can replace one without breaking the others.

That is why we mentioned “A harness that compiles itself” in the title.

Since every candidate is type-checked before it runs, a malformed edit fails at assembly instead of mid-task.

Across five benchmarks and three model families, evolved harnesses gained 14.5% on average. The gains scaled inversely with baseline strength.

On ALFWorld, the weakest agent improved by 44%. The same evolution moved a stronger model 11%, and a near-ceiling benchmark moved 1.1%.

An evolved harness recovers behavior that a weak model cannot produce on its own. The weights are fixed throughout. Only the harness changes.

This is the next phase of harness engineering. The field moved from weights to context, to hand-built harnesses, and the harness was the last component still edited by hand.

In case you missed it, we wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent.

​Read it here →​

​And you can read the HarnessX paper here →

Multi-head attention in Transformers​

Before Transformers, models struggled to understand context. Multi-Head Attention changed everything.

How it works:

• Input gets transformed into Queries, Keys, and Values

• Each attention head focuses on different relationships - one captures grammar, another tracks long-distance dependencies

• Attention scores determine which words matter to each other

• Scores get normalized and combined into context vectors

The magic happens when you run multiple heads in parallel. Each head discovers unique patterns, and together they give the model a complete understanding.

This matters because traditional models like RNNs process sequences step-by-step, losing context. Multi-Head Attention sees everything at once and understands relationships that would otherwise be invisible.

This is why modern AI can write code, translate languages, and understand nuance.

For further practical reading, ​​we implemented Llama 4 from scratch, which is a mixture of experts, in this article →​​

It covers:

• Character-level tokenization

• Multi-head self-attention with rotary positional embeddings (RoPE)

• Sparse routing with multiple expert MLPs

• RMSNorm, residuals, and causal masking

• And finally, training and generation.

Thanks for reading!

Adding an LLM call to a product puts security focus on prompt filtering, output guardrails, and input validation. That only covers the application layer.

To understand this, trace a single model call.

The prompt leaves the app, hits a provider endpoint, and if it carries a patient record or financial data, regulated data just left the boundary.

Depending on the provider’s retention policy, it can sit in external logs for days.

Prompt filtering governs what goes into the model. It does not control where data travels or who stores it.

AWS moves this into the infrastructure layer:

• IAM policies scope which services call which endpoints.

• VPC isolation keeps tenant traffic apart.

• Encryption and CloudTrail logging cover every data flow, including model calls, by default.

So a new AI feature inherits all of it on day one.

IDEMIA handles identity data for 45+ US government agencies on AWS. And they cut transformation time by 4x with the security coming from the infrastructure, not the application code.

You can explore AWS’s resources for building AI-native SaaS here →

Thanks to AWS for partnering today!

Hermes Kanban: Mission control for your agents

We built a 4-agent software team where everything runs from Telegram and is managed on a Kanban board.

• A project manager who plans the work

• A backend developer

• A frontend developer

• And a tester.

The PM reads a goal, breaks it into linked tasks, and assigns each to the right agent.

The thing that makes them a team instead of four strangers is a shared kanban board. Every task is a row, and when an agent finishes, it writes a summary of what it built and what the next agent needs to know.

The next agent reads that summary before it starts. So the frontend developer never has to guess the API shape, and the tester knows exactly what to verify.

The hardest part we faced when setting this up was building an agent that could actually act like a backend engineer.

A backend engineer has to stand up a database, wire auth, manage storage, deploy functions, and keep all of it consistent while the rest of the team builds on top.

An agent doing this from scratch failed almost every time in our run.

It burned its context window, remembering which tables existed and which endpoint it created. Due to this, the work context exhausted quickly.

We solved this by adding InsForge as the backend context engineering layer. It is an open-source, agent-native backend, and we added it to our backend developer agent as a skill.

A skill is a step-by-step guide that teaches the agent how to do a specific kind of work.

With InsForge installed, the agent stopped improvising infrastructure and followed a reliable path to create the project, define the database, set up auth, and deploy functions.

To test the whole team, we had them build a working Google Docs clone, AI features included.

The backend agent spun up the full service on its own, like database tables, user auth, document handling, and edge functions running real TypeScript, all in one dashboard.

The frontend agent read that summary and built the UI on top of it, and the tester closed the loop.

InsForge GitHub repo →

(don’t forget to star 🌟)

You can read our full article covering this demo with full setup here on X →

The top Claude Code CLI integrations​

We created a collection of some of the top Claude Code CLI integrations you should add to your Agents:

1. GitHub → Helps the agent read and write issues, PRs, Actions, and releases, so it works the codebase the way an engineer does, not by editing text on disk.

2. HuggingFace → It pulls a base model, runs the training, and pushes the fine-tuned version back, without you ever leaving the terminal. The whole loop happens in one place.

3. Bright Data → It pulls live search, full pages, and clean data from sites that normally block bots, and now it can even build custom scrapers from the terminal. You can collect data from any website by turning prompts into ready-to-run scrapers with built-in proxies and automatic unblocking.

GitHub Repo: ​https://github.com/brightdata/cli​

(don’t forget to star 🌟)

4. Stripe → It forwards live webhooks and fires real payment events, so the agent runs through the whole checkout instead of faking it.

5. InsForge → A full backend in one CLI (also used in the Kanban demo above). It gives database, auth, storage, edge functions, hosting, and an AI gateway...all in one place instead of stitching five services together. The agent sets up the infrastructure the way a backend engineer would.

GitHub: ​https://github.com/InsForge/insforge​

(don’t forget to star 🌟)

6. CodeRabbit → It reviews the agent’s own code before you ever see it. It catches bugs, security holes, and sloppy patterns while the change is still local, so nothing messy makes it into a PR.

7. Playwright → It allows the agent to click, fill forms, take screenshots, and run UI tests across Chrome, Firefox, and Safari, on the real page instead of guessing from the HTML.

8. Google Workspace → Connect Gmail, Drive, Calendar, Sheets, and Docs through one connector. It is built on Google’s own APIs and made for agents to actually do the work, not just read it, so it can draft the reply, update the sheet, and block off the calendar in one go.

9. Slack → It puts the agent right where your team already works. It builds and runs workflows that post updates and sort through channels.

10. E2B → A safe sandbox for code that the agent wrote itself. It spins up a small isolated VM, runs the code, grabs the output, then shuts the whole thing down.

GitHub: ​https://github.com/e2b-dev/E2B​

(don’t forget to star 🌟)

11. Unsloth → It gives the agent fast local fine-tuning without the cloud bill. It trains LoRA and QLoRA adapters about 2x faster on a lot less VRAM, then exports to GGUF or pushes straight to the hub.

GitHub: ​https://github.com/unslothai/unsloth​

(don’t forget to star 🌟)

12. ffmpeg → This lets the agent cut, convert, and pull audio or video out of just about anything in a single command.

👉 Over to you: What popular CLI integrations do you use?

That said, if you want to see how this whole stack fits together, we wrote a full deep dive on how Claude Code’s harness works, what actually goes in the .claude/ folder, and how hooks, skills, and subagents come together into a real workflow.

​Read it here →

Good day!

P.S. For those wanting to develop “Industry ML” expertise:

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?

• Drive revenue?

• Can you scale ML models?

• Predict trends before they happen?

We have discussed several other topics (with implementations) that align with such topics.

Develop "Industry ML" Skills

Here are some of them:

• Learn everything about MCPs in this ​crash course with 9 parts →​

• Learn how to build Agentic systems in a crash course with 14 parts.

• Learn how to build real-world RAG apps and evaluate and scale them in this crash course.

• Learn sophisticated graph architectures and how to train them on graph data.

• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.

• Learn how to run large models on small devices using ​Quantization techniques.

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using ​Conformal Predictions.

• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.

• Learn how to scale and implement ML model training in this practical guide.

• Learn techniques to reliably test new models in production.

• Learn how to build privacy-first ML systems using ​Federated Learning.

• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

Mythos got shut down this week.

A company built on top of intelligence it didn’t control suddenly found itself exposed to decisions it couldn’t influence.

That’s what the cost debate kept missing. For two years, the only question about open models was whether they were cheaper, but cheaper tokens don’t help you the day the platform changes the rules underneath you.

And to be fair, nobody fully owns an open model either. The base weights still come from labs you don’t control.

The difference is that once those weights are on your hardware, they can’t be taken back. A frontier API can drop you tomorrow, but the weights you already have stay yours.

The base model was never really the point anyway. The real value is in the post-training, your data, your evals, the behavior shaped around the work only your company does.

We read a good piece on this on X. Worth a read if Mythos has you rethinking what you’re actually renting →

[Hands-on] Build a real-time satellite tracker with Claude Code

We gave Claude Code one prompt, and it built a real-time satellite tracker that shows every active satellite orbiting Earth on an interactive 3D globe:

There are over 10,000 active satellites up there right now, including 6,000+ Starlink satellites.

The tracker visualizes all of them moving along their real orbital paths, with a timeline slider that lets you drag from 2000 to 2026 and watch space gradually fill up as satellites launched over the years.

The backend runs on Tiger Cloud by Tiger Data, which gives you managed TimescaleDB on Postgres.

Claude Code provisioned the database, created the schema, seeded the data, and built the frontend in a single session. Let’s walk through the setup, the prompt, and what we built.

Setting up the backend

The entire database infrastructure was provisioned from inside Claude Code using the Tiger CLI MCP server. These are three commands to get started:

# Install Tiger CLI
curl -fsSL https://cli.tigerdata.com | sh

# Authenticate with Tiger Cloud
tiger auth login

# Connect Claude Code to Tiger Cloud via MCP
tiger mcp install claude-code

After restarting Claude Code, it has full access to Tiger Cloud through the MCP server: creating services, running SQL, and querying documentation.

The prompt

The prompt we gave Claude Code covered the full stack in one shot:

Build a real-time satellite tracker with an interactive 3D globe 
showing satellites orbiting Earth.

Backend: Use our Tiger Cloud account (via Tiger MCP) to create a 
TimescaleDB service. Create a hypertable for satellite position 
snapshots (norad_id, object_name, country, launch_date, latitude, 
longitude, altitude_km, velocity, timestamp). Store snapshots for 
thousands of satellites every 5 minutes. Set up continuous aggregates 
for orbital statistics per constellation per hour.

Data: Pull satellite data from CelesTrak public orbital database. 
Compute real-time positions using satellite.js from TLE data.

Frontend: Next.js + Three.js. Dark theme. The globe should have a 
day/night cycle with NASA satellite imagery, city lights on the dark 
side. Each satellite rendered as a glowing dot colored by constellation
(Starlink: cyan, GPS: gold, Weather: green, ISS: red). Add a timeline
slider from 2000 to 2026 that filters satellites by launch date. 
Clicking any satellite shows its name, launch date, country, altitude, 
velocity, and orbital parameters.

After this, Claude Code provisioned the TimescaleDB service, created the hypertable, set up continuous aggregates, pulled orbital data for 10,000+ satellites, computed their positions, and built the complete Next.js frontend with the Three.js globe.

Tracking thousands of satellites every 5 minutes generates a lot of time-series data fast, and that’s what TimescaleDB handles best.

Speedcast already does this at production scale, monitoring 12,000+ Starlink terminals and ingesting 20GB/hour of satellite telemetry through Tiger Cloud. Learn more here →

Its hypertables partition the data by timestamp automatically, so each query only hits the relevant time chunk.

Continuous aggregates serve queries like “average Starlink altitude over 24 hours” from pre-computed summaries instead of recalculating from scratch every time.

On a regular Postgres table, this would require manual partitioning and index tuning to stay fast as the position snapshots accumulate. TimescaleDB handles this out of the box.

To get started:

# 1. Sign up for Tiger Cloud (link below)

# 2. Install Tiger CLI
curl -fsSL https://cli.tigerdata.com | sh

# 3. Authenticate
tiger auth login

# 4. Connect Claude Code
tiger mcp install claude-code

# 5. Give Claude Code a prompt and let it build

Tiger Cloud gives new users $1,000 in free 30-day credit, no credit card required. That’s enough to run a production TimescaleDB instance for a full month and build whatever you want on top of it.

You can sign up for Tiger Cloud here →

Tiger CLI is open-source (Apache 2.0) and works with Claude Code, Cursor, Codex, Gemini CLI, and VS Code.

👉 Over to you: what would you build if Claude Code could provision your database infrastructure?

Thanks to Tiger Data for working with us on today’s issue!

From AGI → ASI​

There’s a brilliant new paper from Google DeepMind called “From AGI to ASI,” and it skips the fight everyone else is having.

Almost every AI debate today is about reaching human-level intelligence. This one asks the harder question instead.

Once we have an AI as capable as a person, where does it go next?

Their reasoning is uncomfortable because it’s so simple.

Essentially, the moment you have one human-level AI, you can copy it exactly, down to its memory and everything it has ever learned.

So you don’t end up with one digital worker but rather a million of them, running faster than any human and sharing every lesson instantly.

A group like that could outthink any company or research lab long before any single AI becomes a lone genius.

That’s the heart of it. Scale alone might carry us from “as smart as a person” to “smarter than entire teams of the best experts.”

The paper maps four ways this could play out, and they can all happen at once.

Keep scaling what already works, i.e., invent a new approach when the current one runs dry, let AI improve AI in a loop that feeds itself. or let armies of AIs organize into something smarter than any one of them, the way a company outperforms any single employee.

They also list everything that could slow it all down, like running out of good training data, maybe costs and energy becoming impossible to sustain, or research simply getting harder as the easy wins disappear.

But the most interesting roadblock isn’t a wall but rather a question.

Today’s AI learns from everything humans have already figured out, and it’s brilliant at remixing our ideas. What it has never done is invent a genuinely new idea from scratch.

A test by Demis Hassabis is a good criterion, which drops an AI into the year 1900 with everything Einstein knew, and asks it to discover relativity. Right now, it can’t, and something is clearly still missing.

The popular picture is one big moment where everything changes overnight. This paper argues that there will be a series of waves instead, each one reshaping a field of science or the economy, arriving faster than the last.

And the most useful work right now involves building better ways to measure progress, so we actually notice which wave we’re already in.

​You can find it here →

Good day!

P.S. For those wanting to develop “Industry ML” expertise:

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?

• Drive revenue?

• Can you scale ML models?

• Predict trends before they happen?

We have discussed several other topics (with implementations) that align with such topics.

Develop "Industry ML" Skills

Here are some of them:

• Learn everything about MCPs in this ​crash course with 9 parts →​

• Learn how to build Agentic systems in a crash course with 14 parts.

• Learn how to build real-world RAG apps and evaluate and scale them in this crash course.

• Learn sophisticated graph architectures and how to train them on graph data.

• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.

• Learn how to run large models on small devices using ​Quantization techniques.

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using ​Conformal Predictions.

• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.

• Learn how to scale and implement ML model training in this practical guide.

• Learn techniques to reliably test new models in production.

• Learn how to build privacy-first ML systems using ​Federated Learning.

• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

95% of enterprise AI projects fail to deliver, and the technology is rarely the problem.

Governance gaps, an inability to justify ROI to finance, and deployment patterns that weren’t built for regulated environments are where things break down.

The AI Strategy Blueprint distills seven years of Fortune 500 and government AI deployments into a book practitioners can actually follow:

Get your copy for free

• The 10-20-70 rule for AI investment

• Governance frameworks that accelerate adoption

• ROI methods that hold up under CFO scrutiny

• Deployment patterns for regulated and air-gapped environments

• A maturity roadmap from crawl to scale

• Case studies from deployments that shipped

Through our partnership with Iternal Technologies, DailyDoseofDS readers can claim the full book for FREE (PDF or Kindle).

The offer is only valid for the next 72 hours.

Grab your free copy here before time ends →

Get your copy for free

​Deep dive on proximal policy optimization (PPO) in RL

Part 8 of the Reinforcement Learning course is available now.

It covers PPO, the algorithm that laid the foundation for everything we are seeing right now in LLM alignment and modern RL. RLHF for ChatGPT was built on PPO.

GRPO, DPO, and every major alternative that followed were designed in direct response to it.

Reinforcement Learning Nanodegree Part 8

You can read Part 8 of the course here →

It covers:

• Why large policy updates can cause irreversible collapse

• How trust regions keep updates safe

• The clipped surrogate objective

• How the full PPO algorithm works

• The KL-penalty variant used in LLM alignment like PPO

• Practical diagnostics for spotting unhealthy training runs

• A from-scratch implementation trained on LunarLander

• How PPO connects to RLHF for language models

Everything is covered from scratch, so no RL background is required.

You can read Part 8 of the course here →

Reinforcement Learning Nanodegree Part 8

Why care?

PPO is the algorithm that started the current era of LLM alignment.

When OpenAI first aligned language models with human preferences, the RL algorithm underneath was PPO.

Every major alternative that came after was designed with PPO as the reference point.

• DPO was introduced specifically to avoid the complexity of running PPO’s RL loop.

• GRPO modified the advantage estimation to remove the need for a learned critic.

• Constitutional AI restructured the reward signal but kept the same trust-region optimization underneath.

You cannot fully understand any of these methods without understanding the algorithm they are all responding to.

Beyond language models, PPO remains the go-to algorithm across robotics, game-playing, and agentic systems.

It earned that position by being simple enough to implement in a few hundred lines of PyTorch and robust enough to handle a remarkably wide range of problems with minimal tuning.

This chapter is also the convergence point of the series. Every idea we have covered so far (value functions, policy gradients, actor-critic, GAE) comes together here into one working system.

Here’s what we have covered so far:

• Part 1: Foundations of Reinforcement Learning →

• Part 2: Markov Decision Processes and Value Functions →

• Part 3: Bellman Equations and Dynamic Programming →

• Part 4: Model-Free Learning →

• Part 5: Function Approximation →

• Part 6: Introduction to Deep RL and DQN →

• Part 7: Policy Gradients, REINFORCE, and Actor-Critic →

• Part 8: Proximal Policy Optimization in RL →

Just like the MLOps course, each chapter will clearly explain necessary concepts, provide examples, diagrams, and implementations.

👉 Over to you: What topics would you like us to cover in this RL series?

Thanks for reading!

The entire reasoning model space has been moving in one direction, emphasizing more thinking tokens, longer chains, and bigger reasoning budgets.

Kimi just challenged that.

K2.7 Code scores higher than K2.6 on every coding benchmark while using 30% fewer thinking tokens, at the same price.

Most thinking models apply the same reasoning depth to every task. A simple bug fix triggers the same deliberation loop as a complex architecture decision. The model isn’t being thorough but rather overthinking.

K2.7 targets this specifically. Here’s what changed vs K2.6:

• +21.8% on Kimi Code Bench v2

• +11.0% on Program Bench

• +31.5% on MLS Bench Lite

All with 30% fewer thinking tokens.

You can find the model on HuggingFace here →

Schema-guided Agent Memory

Most agent memory systems optimize for recall. The harder problem is what to forget, or more precisely, what to never store in the first place.

The default agent memory pipeline hands an LLM raw text and asks it to extract entities and relationships. The model decides the types, labels, and attributes all on its own.

The result is a knowledge graph that behaves like an expensive vector store. Entity types collapse into generic labels. Relationships flatten into a single “RELATES_TO.”

The graph has the data, but no query can reach it with precision.

This is not a retrieval problem but rather a structure problem. And the fix is the same pattern that already works everywhere else in the AI stack, i.e., constrain the output space before generation, not after.

• Entities define what the agent is allowed to remember. Pydantic models with typed fields and descriptive docstrings replace the LLM’s guesswork with domain vocabulary it was never trained on.

• Edges define how things connect. Source/target constraints on relationship types mean the graph can only form valid connections. If your schema has no edge connecting Project to Competitor, that relationship cannot exist in memory.

• Temporal resolution handles what was true versus what is true. Fact resolution invalidates outdated edges while preserving history, so the graph never silently serves a stale state.

The schema guides extraction at two points in the pipeline (entity extraction and fact extraction) while resolution and temporal processing run automatically downstream.

You define what to look for. The system handles deduplication, contradiction detection, and time-windowing without additional configuration.

A useful constraint is to have just 10 entity types, 10 edge types, and 10 fields per type.

That forces you to model the 80% that matters rather than attempting completeness. Start with 3-4 of each and expand only when retrieval fails.

Zep Graphiti does all of this as a fully open-source temporal knowledge graph library.

It includes Pydantic-based ontology definition, schema-guided extraction, entity resolution, fact resolution, and temporal windowing.

If you are building agent memory with any kind of domain specificity, schemas are a must.

Here’s the GitHub repo →

(don’t forget to star 🌟)

We also covered this topic in more depth (with code) in this article →

Thanks for reading!

P.S. For those wanting to develop “Industry ML” expertise:

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?

• Drive revenue?

• Can you scale ML models?

• Predict trends before they happen?

We have discussed several other topics (with implementations) that align with such topics.

Develop "Industry ML" Skills

Here are some of them:

• Learn everything about MCPs in this ​crash course with 9 parts →​

• Learn how to build Agentic systems in a crash course with 14 parts.

• Learn how to build real-world RAG apps and evaluate and scale them in this crash course.

• Learn sophisticated graph architectures and how to train them on graph data.

• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.

• Learn how to run large models on small devices using ​Quantization techniques.

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using ​Conformal Predictions.

• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.

• Learn how to scale and implement ML model training in this practical guide.

• Learn techniques to reliably test new models in production.

• Learn how to build privacy-first ML systems using ​Federated Learning.

• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

We spent a morning with Claude Fable 5, and it left us thinking about one thing, i.e., what an agent is allowed to do while it runs on its own.

Fable 5 runs for hours, holds a goal for days, and takes action without checking in. So the question stopped being what an agent can build. It became what it can touch while it runs.

We tested this with a plain internal tool.

Fable 5 wrote the SQL, pulled real records, and added a button that issued a refund straight to the database, all in one pass.

The build was easy, but deciding who can open it, who can run that refund, and keeping a record of what happened was the part the model could not handle on its own.

That part lives in the runtime, not the model.

We deployed the same agent into Retool, and it picked up company SSO, a role check on the refund, and an audit log of every query and action. The app didn’t change, but where it ran did.

Retool calls this building inside the lines, and you’ll notice the difference once you watch the same agent run both ways.

You can try it out yourself here →

Thanks to Retool for partnering today!

AI concepts every AI engineer should know

Two teams can build on the same base model and ship completely different products.

The model is a fixed input. What differs is the eight layers of engineering wrapped around it, from how tokens get served to how the agent loop is controlled.

We mapped the full set into a single diagram:

Model foundations cover how a model turns text into probabilities:

• Tokenization splits text into subword unit before the model ever sees it, and token count drives both cost and context limits.

• Embeddings map those tokens to high-dimensional vectors where similar meanings sit close together.

• Pretraining learns language from raw text while post-training (SFT, RLHF) shapes behavior and alignment.

• The context window is the fixed token budget the model attends to at once, shared across prompt, history, and output.

• Logits are the raw scores over the vocabulary that sampling (explained below) turns into the actual output token.

• Sampling controls how the next token is drawn from the distribution, with temperature and top-p trading determinism for diversity.

We covered this layer in depth (with code) here →

Inference and serving cover the stack that turns weights into cheap/fast tokens:

• Prefill processes the whole prompt in parallel and is compute-bound, while decode generates one token at a time and is memory-bound.

• The KV cache stores attention keys and values for past tokens so they aren’t recomputed at every step.

• Prompt and prefix caching reuses the KV state of a shared prefix, which makes fixed system prompts effectively free after the first call.

• Speculative decoding uses a small draft model to propose several tokens that the main model verifies in parallel for a net speedup.

• Continuous batching fills GPU slots with new requests the moment others finish, instead of waiting for a whole batch to complete.

• Quantization stores weights in fewer bits (FP8, AWQ) to cut memory and speed up compute, with FP8 running natively on recent GPUs.

• Paged Attention applies OS-style paging to the KV cache to remove fragmentation, and it’s the core trick behind vLLM.

• TTFT and TPOT measure prefill and decode speed respectively, and both trade against raw throughput.

We covered this layer in depth (with code) here →

Context engineering involves managing what the model sees at the moment it acts:

• Context budgeting treats the window as a finite resource and spends tokens only on what raises answer quality.

• Context rot is the way output quality degrades as the window fills, often well before the hard limit.

• Lost in the middle describes how models attend most to the start and end of the context, so details buried in the middle get underweighted.

• Compaction and summarization condense old history into a high-fidelity summary so the agent can continue in a fresh window.

• Context offloading pushes large details to external files or stores and keeps only a reference in the window.

• Just-in-time retrieval loads data at the step that needs it rather than front-loading everything upfront.

• Structured note-taking lets the agent write persistent notes outside the window and read them back as needed.

We covered this layer in depth (with code) here →

Agents and harness engineering a stateless model into something that finishes tasks.

• The agent loop runs the think, act, observe cycle (ReAct or TAO) until the task is done.

• Tool use and function calling let the model emit structured calls that the harness executes, feeding the results back to the model.

• A thin harness trusts the model and keeps infrastructure minimal, while a thick harness encodes control in code and leaves less to the model.

• Subagents and orchestration spin off focused agents with their own context so the main agent stays lean.

• MCP is a standard interface connecting models to tools and data, replacing N×M custom integrations.

• Skills, hooks, and state add reusable capabilities, lifecycle triggers, and persisted memory that survive across steps.

• Planning versus reacting is the choice between committing to a plan upfront and deciding each step live, with different speed and cost.

• Verification loops use rules, tests, or an LLM judge to gate the agent’s output before it counts as done.

We covered the Anatomy of Agent Harness here →

Retrieval and memory feed the model facts it was never trained on.

• RAG pipelines retrieve relevant chunks at query time and add them to the prompt before generation.

• Chunking and re-ranking split documents into units and then reorder retrieved candidates by true relevance.

• A vector DB stores embeddings and serves nearest-neighbor search at scale.

• Hybrid search combines keyword and vector matching to catch both exact terms and semantic matches.

• Knowledge graphs model entities and relationships so retrieval can follow connections, not just similarity.

• Episodic and temporal memory keep what happened and when, so the agent recalls past sessions and how facts changed.

• Agentic retrieval lets the agent issue and refine its own queries instead of running a single fixed lookup.

We covered this layer in depth (with code) here →

Adaptation and training change the weights when prompting and context aren’t enough:

• Fine-tuning (SFT) trains the model on input-output examples to teach format and behavior.

• LoRA and QLoRA (PEFT) train small adapter weights instead of the full model, which cuts cost and memory sharply.

• RLHF and DPO optimize against human preferences, with DPO skipping the separate reward model.

• GRPO and reward models train against a reward signal, which works when you have a scoring function but no preference labels.

• Distillation trains a smaller student to match a larger teacher, keeping most of the accuracy at lower cost.

• Synthetic data generates training examples with a model when real labeled data is scarce.

We covered this layer in depth (with code) here →

Evaluation and observability help track whether a change helped or quietly broke something.

• Offline and online evals run fixed test sets before shipping, and live metrics from real traffic after.

• LLM-as-judge uses a model to score open-ended output that rules can’t grade.

• Agent trajectory eval judges the full path an agent took, not just the final answer.

• Tracing and spans record each step, tool call, and token so you can see where a run went wrong.

• Token and cost tracking attributes spend per request and per step to find what’s expensive.

• Regression testing re-runs a benchmark after a prompt or model change to catch silent breakage.

We covered this layer in depth (with code) here →

Safety, security and reliability keep the system honest in front of real users.

• Prompt injection is untrusted input that smuggles in instructions to hijack the model’s behavior.

• Jailbreaks are crafted prompts that get the model to bypass its own safety constraints.

• Hallucination mitigation uses grounding, retrieval, and verification to catch confident wrong answers.

• Structured outputs constrain generation to valid JSON or a schema so downstream code can parse it.

• PII and data privacy controls detect and redact sensitive data before it enters or leaves the model.

• Fallbacks and rate limits add multi-provider failover and request caps that keep the system up under load.

• Guardrails are input and output filters that block unsafe or off-policy requests and responses.

We covered this layer in depth (with code) here →

The inference layer is where most production cost hides.

On an H100 running Llama 70B, a single request hits 92% GPU utilization during prefill, then drops to 28% during decode on the same hardware a moment later. The workload changed, not the GPU.

No single technique moves that number much. Stacking eight or nine of them, across compression, attention, decoding, caching, and routing, is what closes a 5-8x cost gap against naive FP16 inference.

We broke down all 72 of these techniques, with the engineering logic behind each, in this issue and across the full LLMOps crash course.

For routing trivial queries to cheaper models, the advisor strategy is another interesting path to start with.

👉 Over to you: which of the eight layers do you spend the most time in?

Thanks for reading!

They tracked 807 GitHub repos that adopted Cursor.

Each was matched against a similar repo that did not, so the difference reflects the agent, not the project.

In the first month after adoption, the agent-assisted repos added 3 to 5x more code. That gain was real but short-lived, and it faded within two months.

What did not fade was the hit to code quality.

Static analysis warnings rose around 30% and code complexity around 41%, and both stayed elevated for the rest of the study.

The obvious objection is that this is a skill problem, fixable with sharper prompts and review.

But even after controlling for how much code was added, complexity still climbed faster in the agent-assisted repos than in the controls. The agent adds to the problem, not just the developer using it.

The instinct is to have the agent review the code it just wrote, or add a second agent to check it.

That does not work as well as expected because the model that wrote the code carries the same blind spots that produced the defects.

A second model trained on similar data shares most of them, so an AI reviewer misses what the AI author missed.

Catching this reliably needs deterministic code analysis, which applies fixed rules instead of another probabilistic guess.

SonarQube actually implements this as a plugin inside Claude Code.

It runs deterministic verification in the session, flagging issues, complexity, and exposed secrets after every edit across the project.

For instance, in the video below, I typed a prompt with a GitHub token in it, and SonarQube blocked it before it reached Claude. The token never entered the context window.

To get started in Claude Code, run /plugin, find sonarqube, install it, then run /sonarqube:integrate.

You can find the Claude plugin here →

And here’s the CMU’s study →

Thanks to Sonar for partnering with us!

Training an LLM to generate reliable structured output

An LLM has no inherent concept of a parser. It predicts the next token from the ones before it, and well-formed JSON is only one of the continuations that fit this format.

So a number written as a string fits just as well, and so does an object with a line of explanation in front of it.

More examples and a tighter prompt do push the failure rate down, but then it stalls.

DeepSeek-R1 showed a way around this. Training a strong model used to mean annotation pipelines, preference pairs, and a team of labelers.

DeepSeek replaced all of it with one function that checks whether an answer is right. If you can define correctness via code, you do not need the rest.

That is the idea behind GRPO. Instead of learning from examples, the model learns from a reward function you write.

For each prompt, it generates a few candidate answers. The reward function scores them, and the model is pushed toward the ones that score higher.

Today, let’s use this idea to fine-tune Qwen3-8B for JSON extraction. The loop runs from a local notebook while the model trains on remote H200s.

The reward function just checks whether each output parses and matches the schema.

Schema-valid output went from 62% on the base model to 82% after training, past GPT-4.1 on the same eval at 58%.

Before we dive into code, let’s see why the obvious approach stalls.

Problems with SFT

SFT learns by copying examples. If you show it correct completions, it gets good at producing output that looks like them.

Notice the word “looks.”

This implies that looking like valid JSON and being valid JSON are different goals, and SFT only optimizes the first one.

The loss is measured token by token, and a completion with one field typed wrong scores almost the same as a perfect one.

If you add more examples, the metrics do tick up, but then they flatten, because the limit is the objective, not the data.

Once you see the problem this way, the fix becomes immediately clear. You can define what “correct” means in code, and train the model against that definition directly.

This is what GRPO does.

It swaps labeled examples for a reward function. For each prompt, the model generates a small group of answers (usually 4-8). And then the reward function scores every one of them.

The scores are normalized inside the group. The model update then reinforces the answers that scored above the group’s average.

So the model is always comparing its own outputs against each other. It learns what “more correct” means for your task, not what “more similar to an example” means.

Here is how the reward function scores three different outputs for the same prompt:

• Output that doesn’t parse as JSON scores 0.0.

• Output that parses but fails the schema scores 0.5.

• Output that parses and matches the schema scores 1.0.

That middle score is important because without it, valid JSON with the wrong field types would score zero, the same as complete garbage.

The model would lose an important signal that a valid structure is already progressing. The 0.5 is what gives training something to climb toward.

Infra for GRPO

GRPO is heavier than SFT. On an 8B model, it needs H200s and runs for hours.

Every step, it generates several completions per prompt, scores all of them, and updates the weights. That repeats across the whole dataset, many times over.

So this is not something you can run on your laptop.

There is also a timing problem that SFT never has.

During rollout, the model samples answers from its current weights. During training, those same weights are changing underneath it.

If the inference side and the trainer fall out of sync, you sample from a stale model and train on answers that the current one would never give. This is where most custom RL setups fall apart.

Fireworks’ Training API handles both sides. You can write the training logic in Python on your own machine, and the infrastructure handles the rest.

It provisions the GPUs, runs the forward and backward passes, saves checkpoints, and resyncs the inference deployment after every step.

The setup is three steps:

• Write the reward function

• Upload the dataset

• And configure the run.

Let’s go through each one.

The training loop

Fireworks documents the full setup in their Training API docs. That includes rl_loop, the recipe that runs the entire GRPO loop.

Clone the cookbook and install the training dependencies:

git clone https://github.com/fw-ai/cookbook.git

cd cookbook/training && pip install -e ".[training]"

export FIREWORKS_API_KEY="your-training-scoped-key"

Step 1: Write the Reward Function

This is the only place your task is defined. The SCHEMA says what a correct output looks like, and score() checks each completion against it.

For invoice extraction, we pull four fields from raw text: vendor, date, amount, and currency.

import json
from jsonschema import validate, ValidationError

SCHEMA = {
    "type": "object",
    "required": ["vendor", "date", "amount", "currency"],
    "properties": {
        "vendor":   {"type": "string"},
        "date":     {"type": "string"},
        "amount":   {"type": "number"},
        "currency": {"type": "string"},
    },
    "additionalProperties": False
}

def score(completion: str) -> float:
    try:
        parsed = json.loads(completion.strip())
    except (json.JSONDecodeError, ValueError):
        return 0.0
    try:
        validate(instance=parsed, schema=SCHEMA)
        return 1.0
    except ValidationError:
        return 0.5

jsonschema handles the type checks, the required fields, and any nested rules in a single call.

For a different task, like SQL or tool-call formatting, you can swap in a new schema. The score() function stays the same.

Step 2: Prepare the Dataset

GRPO does not need labeled outputs. The dataset is just the prompts you would send in production.

The model writes its own completions during training, and score() grades them as they come.

We used 200 training prompts. They cover different vendor names, date formats, amount styles, and currency codes. We also set aside 50 eval prompts.

Variety matters more than volume here. Prompts that all look alike produce a model that breaks on real invoice variation.

{
  "messages": [
    {
      "role": "user",
      "content": "Extract the following fields from this invoice:
                  Bill from Acme Corp, dated 2024-03-15, total $1,250.00 USD.
                  Return valid JSON only."
    }
  ]
}
{
  "messages": [
    {
      "role": "user",
      "content": "Extract the following fields from this invoice:
                  Received from TechSupplies Inc on January 8 2024,
                  amount due: 340 euros. Return valid JSON only."
    }
  ]
}

Upload the dataset and wait for the READY state before the training job can use it.

from fireworks import Fireworks

client = Fireworks()
client.datasets.create(dataset_id="invoice-extraction-grpo-v1",
                       dataset={"exampleCount": "200"})
client.datasets.upload(dataset_id="invoice-extraction-grpo-v1",
                       file="./train_prompts.jsonl")

# poll client.datasets.get(...).state until "READY" before proceeding

Step 3: Configure and Run the Loop

rl_loop runs the whole thing. It provisions the trainer, schedules the rollouts, syncs the weights, and cleans up when the run is done.

You connect your score() function by wrapping it and assigning it to rl_loop.reward_fn. The wrapper gets both the completion and the dataset row, so you can reach ground-truth metadata if your reward needs it.

from training.recipes.rl_loop import Config, main
from training.utils import DeployConfig, InfraConfig, WeightSyncConfig
import training.recipes.rl_loop as rl_loop

# Wire your reward function to the training loop
def invoice_reward(completion: str, row: dict) -> float:
    return score(completion)

rl_loop.reward_fn = invoice_reward

cfg = Config(
    base_model="accounts/fireworks/models/qwen3-8b",
    dataset="invoice-extraction-grpo-v1",
    max_rows=200,
    epochs=1,
    completions_per_prompt=4,
    max_completion_tokens=256,
    temperature=1.0,
    max_seq_len=4096,
    policy_loss="grpo",
    output_model_id="invoice-extractor-v1",
    infra=InfraConfig(
        training_shape_id="accounts/fireworks/trainingShapes/qwen3-8b-128k",
    ),
    deployment=DeployConfig(
        deployment_id="invoice-extractor-v1",
        tokenizer_model="Qwen/Qwen3-8B",
    ),
    weight_sync=WeightSyncConfig(weight_sync_interval=1),
)

main(cfg)

A few of these settings to note here:

• dataset points to the dataset_id uploaded in Step 2. Fireworks pulls it from their storage directly.

• completions_per_prompt=4 sets the group size for GRPO. Production runs often use 8 to 16, which gives more signal per step at a higher compute cost. Four is enough here. The reward is clear-cut enough that even a small group shows real variance between answers.

• weight_sync_interval=1 resyncs the inference deployment after every step. That keeps rollout sampling from the exact model being trained. Production runs often set this to 4 or 8 for speed. For a short 200-step run, 1 gives the tightest feedback loop, which is what you want.

• One Qwen3 quirk to handle here is that it defaults to thinking mode and adds <think>...</think> blocks before the answer. Strip them at eval time with content.split(”</think>”)[-1].strip(). Suppress them in training by adding /no-think to the system prompt. Otherwise, the reward function reads the reasoning text instead of the JSON, and scores everything 0.0.

Results on the held-out eval

Base Qwen3-8B scores 62% schema-valid on the 50 held-out prompts. After GRPO training on Fireworks H200s, the fine-tuned model hits 82%.

That is past GPT-4.1 on the same eval, which lands at 58%.

Here is the baseline run first, on the 50 prompts the model never trained on.

And here is the same eval after training.

The trained model runs on a Fireworks serverless endpoint, at a fraction of GPT-4.1’s per-token cost. Latency is lower too, since the output is short and predictable.

The real difference shows up on messy inputs. A prompted general-purpose model starts to slip, while the trained model holds, because it learned the constraint instead of the examples.

This pattern works for any task where you can check correctness in code.

SQL that has to parse, API responses that must match a shape, tool calls, and code that has to pass a linter.

If you can score an output, you can train a model to chase that score.

What DeepSeek-R1 proved at the frontier scale holds for your own small task. The model you get has practiced your definition of correct, not memorized examples of it.

You can find the training API docs in this GitHub repo →

The full code is in the repo below. That includes the reward function, the training config, the dataset builder, and the eval script.

And you can find the finetuning code in our repo here →

Thanks for reading!

Your first agent: running in a few lines of code.

Your production agent: the same lines of code, scaled.

Strands grows with your use case, from a local prototype to a deployed system handling real workloads, without a rewrite in between.

Open source. Model-driven. Any backend.

Start building →

Thanks to AWS for partnering today!

Loop engineering: Design the system that prompts agents

In a hand-run agent session, you hold two jobs:

• deciding what the agent runs next

• and checking its output before the next step.

Loop engineering moves both into the system.

Essentially, a schedule decides what runs, a separate checker agent grades the output, and a file on disk holds the state that they both read.

Boris Cherny, who leads Claude Code, stopped prompting Claude by hand and now writes the loops that prompt it instead.

So today, let’s break down what a loop is built from, the split that does the real work, and what stays manual.

The six parts a loop are built from

A loop is assembled from five components plus a memory, and Claude Code and Codex both ship all of them now.

• Automations fire on a schedule and do discovery and triage on their own.

• Worktrees give each agent its own checkout, so two running in parallel don’t overwrite each other’s files. Leaving orphaned worktrees behind is the main way this turns messy.

• Skills hold the project knowledge the agent would otherwise re-derive from zero every run, like conventions, build commands, and review standards. An automation can fire a skill by name instead of carrying a wall of instructions that nobody maintains.

• Connectors wire the agent into your real tools over MCP, the issue tracker, the database, Slack.

• Sub-agents split the work, so one agent makes the change and a different one checks it.

• Memory is the sixth piece, giving you a dedicated knowledge graph (using Zep Graphiti, for instance) that lives outside the conversation and records what’s done and what’s next.

Here’s what you need to make sure when building these systems:

The maker-checker split

The split between the maker and the checker makes a loop safe to leave alone.

This is important because a model grading its own output is too lenient to catch its own bugs, so the checker runs as a separate agent, ideally a different model, scored against independent signals.

Its findings go back to the maker as the next instruction, and the cycle repeats until they come back clean.

Set the exit before you start

A loop with no stop condition churns and burns tokens, and the token cost climbs fast once sub-agents and long runs stack up.

One thing you can do in practice goes something like…fix major issues only, run one final pass, then stop after two loops, with “all tests pass and lint clean” as the explicit exit.

Decide this before the loop runs, not while it’s running.

State lives on disk, not in context

The model forgets everything between runs, so the state has to sit outside the conversation.

A markdown file or a knowledge graph can hold what’s done and what’s still open, and each run reads it first and writes back to it last.

The agent forgets between runs, but this does not, which is precisely what lets a loop pick up across days instead of resetting every session.

Loops are important on the boring, high-frequency checks you skip by eye, like a stale version string or a missing test, but not on work that needs judgment.

An unattended loop fails in two ways.

It reports done when nothing actually verified the work, which is what the separate checker is there to prevent.

And it merges code faster than you read it, so over weeks, you stop understanding your own codebase while every check stays green.

Green tests tell you the code passed the tests, not that you know what shipped, so the habit that keeps a loop honest is reading what it merges, not just watching the checks pass.

We’ll make this much more concrete by sharing a hands-on demo soon.

In the meantime, just remember that loop engineering is a simple trade where you stop being the one who prompts and become the one who designs the loop that prompts it.

👉 Over to you: where would you actually let an agent loop run unattended, and where would you keep your hand on every turn?

Thanks for reading!

P.S. For those wanting to develop “Industry ML” expertise:

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?

• Drive revenue?

• Can you scale ML models?

• Predict trends before they happen?

We have discussed several other topics (with implementations) that align with such topics.

Develop "Industry ML" Skills

Here are some of them:

• Learn everything about MCPs in this ​crash course with 9 parts →​

• Learn how to build Agentic systems in a crash course with 14 parts.

• Learn how to build real-world RAG apps and evaluate and scale them in this crash course.

• Learn sophisticated graph architectures and how to train them on graph data.

• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.

• Learn how to run large models on small devices using ​Quantization techniques.

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using ​Conformal Predictions.

• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.

• Learn how to scale and implement ML model training in this practical guide.

• Learn techniques to reliably test new models in production.

• Learn how to build privacy-first ML systems using ​Federated Learning.

• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

The voice agent stack is one of the few areas in AI where the demand is enormous, and the worked examples barely exist.

Speechmatics Academy has open-sourced a collection of runnable examples across batch, real-time, voice, and TTS, each standalone enough to clone a single folder and have a working pipeline in minutes.

Speechmatics Academy GitHub Repo

The integrations have complete loops with LiveKit, Pipecat, Twilio, and VAPI, covering WebRTC capture, turn detection, speaker focus, interruption handling, and function calling.

The use cases it teaches cover production territory, including SRT captioning, call-center topic detection, and HIPAA-friendly medical microbatching with Silero VAD chunking.

While everyone debates voice as the next interface, this is the repo that shows how to actually build for it.

You can find the GitHub repo here → (don’t forget to star it ⭐️)

Thanks to Speechmatics for partnering today!

Your agent harness should repair itself

When an AI agent fails in production, the observability tool only shows exactly what it did and almost nothing about how to fix it.

You get a clean trace of every model and tool call, the latency per step, and the token usage. What you don't get is why the trace broke, the change that fixes it, or any guarantee that the same failure won't recur.

So you scroll the trace span by span, form a theory, write a patch, and hope it doesn’t break something that already worked.

Then a new model ships with its own failure modes, and you run the loop again from the top.

Cursor recently shared how much engineering goes into the harness around their agent, the prompt layers, tools, and checks wrapped around the raw model. Every model upgrade and every new tool widens that surface.

Today, let’s look at how you can use Opik (open-source) to solve this.

The four-layer stack in Opik

For context, Opik is an open-source logging, debugging, and optimization platform for AI agents and LLM apps.

Opik is built around the premise that the observability loop should be automated, not staffed, which is why Opik’s architecture is one connected workflow.

Trace → Ollie diagnoses → Ollie proposes a fix → fix is applied and verified → Test Suite locks the failure as a regression test → back to Trace

Let’s look at each layer.

Layer 1: Tracing

Every LLM call, tool invocation, and retrieval step is instrumented automatically with a single decorator.

import opik

@opik.track
def my_agent(query: str):
    # your agent logic here
    ...

This works with LangGraph, CrewAI, and 50+ frameworks. Every trace records which agent configuration was active for full reproducibility when you need to rerun a failing input later.

Layer 2: Ollie

Opik’s Ollie is a recent addition that accepts a trace and fixes code. It is a coding agent built into Opik with full context.

If you run it on your project root, Ollie will:

• Read your source files

• Identify the exact lines responsible

• Propose a diff (nothing changes without your explicit approval)

Once you approve, Ollie reruns the agent against the exact inputs from the original failing trace, streams the new trace for side-by-side comparison, and locks the original failure as a regression case in your test suite.

Bad trace → root cause → diff → approve → rerun → regression locked

Without any code access, Ollie reads span trees, identifies failure modes, and explains the causal chain across every LLM call. If you ask it, “why did the final answer ignore the retrieved context?”, it walks the full span tree and surfaces the root cause.

Layer 3: Test suites

Eval workflows typically build a labeled dataset, define a numerical metric, and compare floats, but it doesn’t match how engineers think about quality.

You can replace it with plain-English assertions in Opik:

suite = opik.TestSuite("crm-agent-v2")

suite.add_assertion("The response must include specific deal details, not just a count")

suite.add_assertion("The response must never reveal unauthorized information")

suite.run_tests()

Opik converts those into LLM-as-a-judge checks under the hood and gives clean pass/fail per test case.

Here, every failing trace you debug automatically becomes a new test case, so the test suite naturally grows from real production failures.

Every cycle, the harness gets harder to break.

But even with a growing test suite, you still need a safe place to test changes before they ship. That’s what Layer 4 is for.

Layer 4: Agent sandbox

Most playgrounds are prompt playgrounds. Changing a system prompt and rerunning the LLM call answers the wrong question.

The production question is, what happens to the entire agent graph when I change something?

Opik’s Agent Sandbox runs the fully instrumented agent end-to-end inside the UI. Change a prompt, swap a model, add a tool, and watch how the whole system responds across the full spanning tree.

Every sandbox run produces a complete Opik trace.

In practice

The layers aren’t independent features. They’re one loop that you can instrument with @opik.track.

If something fails in production.

• Ollie reads the trace + source code and proposes a fix.

• You approve.

• Ollie reruns the agent in the Sandbox against the original failing input.

• Once fixes pass, it saves it as a new blueprint.

• The environment pointer promotes to staging.

• Finally, the original failure is locked as a regression test.

The next failure (if any) enters the same loop.

And in every cycle, the harness gets harder to break.

The whole stack we discussed today is open-source (19k+ stars), covering Tracing, Ollie, Test Suites, the Agent Sandbox, a 6-algorithm Agent Optimizer, and 50+ framework integrations.

You can self-host everything in three commands:

git clone https://github.com/comet-ml/opik
cd opik
./opik.sh

Here’s the Opik GitHub Repo → (don’t forget to star 🌟)

👉 Over to you: What’s the current state of observability in your agent stack, and where does the debugging loop break for your team right now?

Good day!