Agents Need a New Kind of Web Search
...covered with three retrieval strategies for AI agents in 2026.
A free O’Reilly book on debugging production systems
Coding agents collapsed the cost of writing code, but validation capacity stayed as is. Tests still encode only the failure modes someone anticipated, and staging still cannot reproduce production traffic.
AI-generated code makes this worse because it fails on unknown-unknowns. Nobody formed a mental model while writing it, so nobody can predict which inputs will break it.
The same holds for prompt changes, which can pass every offline eval and still degrade for one user segment because eval sets rarely match production traffic.
That leaves production as the only environment where this code gets validated, and it demands telemetry that answers questions nobody predicted.
Dashboards only answer the questions someone thought to set up in advance. Catching an unknown failure means keeping raw, high-cardinality events and slicing them by user ID, prompt version, or any other attribute after the fact.”
This is also why Observability Engineering, the 2022 O’Reilly book that became the standard reference on the topic, just got a near-complete rewrite.
The authors rebuilt it around this exact shift, with 27 net-new chapters on instrumenting LLM apps, feeding production telemetry back into evals, and debugging with agentic AI.
Honeycomb partnered with us on this issue and has made the early-release chapters free to download →
Grab the free copy today to learn about observability engineering!
Agents need a new kind of web search
Andrej Karpathy once said LLMs are a bit like a coworker with anterograde amnesia. That’s the condition where you keep your old memories but can’t form new ones.
An LLM lives with that condition in two ways. It doesn’t remember you across conversations, and it never learns anything that happened after its training ended.
Memory features exist to patch the first gap. This piece is about the second one, because agents mostly get hired to deal with the present.
Markets move, people change roles, prices update, news breaks. Web search is how the model sees any of it, and how well it works ends up deciding how useful the whole agent is.
So you bolt a search tool onto your agent and assume it can now read the web. Then you check the first few traces, and the result is disappointing.
Nearly all the tokens you paid for went into raw page text the agent had to dig through. Only a thin slice went into answering your question.
The reason is what a search call actually returns. A typical search API hands back links and thirty-word snippets, nothing more.
So the agent isn’t reading the web, it’s reading a table of contents. Fetching the actual pages becomes its job, which means pulling the HTML, stripping the markup, and digging out the usable text before any real work starts.
On a single query, you barely notice the overhead. In a task that takes several searches, like a research run or a briefing pipeline, you pay it again on every single one.
The problem isn’t that agents search badly. It’s that each search call returns pointers to content instead of the content itself, and the agent pays to close that gap on every call.
We ran the same question through three retrieval setups and counted tokens. The most expensive setup costs roughly 4x the cheapest retrieval option.
This piece walks through those numbers and shows what a search response should look like when an agent is the one reading it. It also covers a class of questions that only full documents can answer.
Let’s start with where the cost actually comes from.
Every loop iteration pays a retrieval cost.
That repeated fetch-and-clean work has a name. Call it the retrieval tax, the tokens an agent burns preparing content before it can reason about your problem.
One looping agent is enough to feel it. A research agent searches, reads what came back, decides what to look up next, and searches again, and every pass pays the tax on top of the last one.
Here’s what that costs on a single question. We picked “What was Y2K?” on purpose, because the model already knows the answer from training.
We answered it three ways, straight from memory, through a web search loop, and through an owned index (a search service that crawls and cleans pages ahead of time and stores its own processed copy of the web), counting the total tokens billed each time.
Picking a question the model already knows is what keeps the experiment clean. The thinking cost is identical in all three runs, so any tokens above it come purely from the retrieval.
From memory, the whole thing took about 600 tokens. That’s the baseline, the cost of just answering with zero retrieval.
An owned index is a search service that has already crawled and cleaned the pages, so one call returns the full document. Going through one took roughly 6,900 tokens.
A single web search hop cost about 3,750 tokens. But snippets are often too thin to answer from, so the agent refetches and refines, and three hops climbed to roughly 28,700, more than 4x the owned index.
The gap between the loop and the index is the tax made visible. Every hop re-billed the entire growing context window, so by the third pass, the agent was mostly paying to carry pages it had already read.
The owned index skipped all of that. The full document was already there when the query arrived.
What makes that possible is what comes back from the search call itself, so let’s look at that next.
Good retrieval returns documents, not directions
The fix isn’t a smarter agent or a better prompt. The fix is what comes back from the search call.
A SERP, the plain list of results a search API returns, hands you URLs and asks the agent to go read the web itself.
A scraper-first tool gets you closer by returning raw HTML or markdown, but the agent still has to clean it before it can use anything.
An owned index does all of that work before the query ever arrives, so what comes back is already readable.
Here’s the same lookup across all three. We’ll search for Christoph Molnar, the author of the Interpretable Machine Learning book.
1) SERP
# query
results = search("Christoph Molnar interpretable ML")
# response
[
{"title": "Interpretable Machine Learning", "url": "christophmolnar.com/books/...", "snippet": "A guide for making black box models explainable..."},
{"title": "Christoph Molnar - Google Scholar", "url": "scholar.google.com/...", "snippet": "Cited by 12,847..."},
# 5 more results across different domains
]The agent got a list of tabs, not an answer.
2) Neural search
# query
results = neural_search("Christoph Molnar", type="person")
# response
[
{"highlight": "Christoph Molnar is a statistician and machine learning interpretability researcher..."},
{"highlight": "Author of Interpretable Machine Learning, an open access book with 400k+ readers..."},
# one stray paragraph about a different researcher
]It found the right person, but as highlight fragments. Getting the full record still needs another call.
3) Owned index
# query
result = client.search("Christoph Molnar", scope="people")
# response
{
"name": "Christoph Molnar",
"roles": [
{"title": "Research Scientist", "org": "Mindful Modeler", "start": "2021"},
{"title": "Author", "work": "Interpretable Machine Learning", "editions": 3}
],
"education": [{"degree": "PhD Statistics", "institution": "LMU Munich"}],
"websites": ["christophmolnar.com", "github.com/christophM"],
"languages": ["German", "English"]
}One call returns the complete record, and the agent goes straight to reasoning.
Seltz is a web index built exactly for this. It crawls the web ahead of time, processes every page into a structured document, and returns finished content through a single API call.
It comes with three scopes, and each one returns a different kind of finished document depending on what your agent needs.
The People Scope returns full structured profiles, with every role and its dates, education, websites, and the organizations someone has worked at.
The News Scope returns complete article text, date-windowed, so you can pull only what was published in a specific period.
The Wiki Scope returns clean Wikipedia documents, useful for building context around companies, industries, or topics before going deeper.
One caveat is worth knowing before you build on the people scope. It works best at the director and regional-leader layer.
For the most senior executives, start with open web search and use Seltz to enrich everyone below them.
Some answers only exist across records
Full documents unlock something that goes beyond cutting the retrieval tax.
Some questions don’t live in any single search result. The answer only exists once you read across multiple records and find where they intersect.
Neither a SERP nor a neural search tool can do this on the first pass. They don’t hold enough of each record to join them, so the work lands back on the agent.
Consider a question a GTM team actually runs. You want every target account that hired someone into a data or AI leadership role in the last quarter.
Answering it needs two things together. Full role histories to find who started a new position and when, plus recent news to confirm the timing and surface the trigger event.
You query the people scope to find who moved into a relevant role recently, then cross-reference the news scope for the trigger event.
What comes out is a ranked list of warm accounts with enough context to write a relevant first outreach line for each one.
That answer doesn’t exist anywhere on the web as a page. It only exists once you join the records, and the join only works when you hold the full records to begin with.
Chain discovery and depth, and every iteration gets cheaper
Seltz isn’t a replacement for open web search. It’s the right tool for a specific class of questions.
Open web search handles discovery well. Finding who currently holds a title, confirming something launched last week, reading a page that went live this morning.
Once you know what you’re looking for, Seltz returns the complete record in one hop rather than a headline and a link.
Many pipelines use the same search tool for every query, regardless of what the question needs. Discovery queries go through the same call as deep-dive lookups, and the loop pays the retrieval tax on both.
Chain the two instead, and each loop iteration gets the right retrieval shape for what it’s actually trying to do.
That’s the core argument here. The cost of an agent loop is set less by the model and more by what each search call hands back, and returning finished documents is how you stop paying for the same pages twice.
If you’re building loops that hit these query shapes, you should give Seltz a try.
Stay tuned as we dig more into efficient web search for Agents and share what we learn along the way.
Thanks to Seltz for working with us on this article.
RAG, Agentic RAG, and AI Memory
RAG was never the end goal.
Memory in AI agents is where everything is heading.
Let’s break down this evolution in the simplest way possible.
RAG (2020-2023):
Retrieve info once, generate response
No decision-making, just fetch and answer
Problem: Often retrieves irrelevant context
Agentic RAG:
Agent decides *if* retrieval is needed
Agent picks *which* source to query
Agent validates *if* results are useful
Problem: Still read-only, can’t learn from interactions
AI Memory:
Reads AND writes to external knowledge
Learns from past conversations
Remembers user preferences, past context
Enables true personalization
The mental model is simple:
RAG: read-only, one-shot
Agentic RAG: read-only via tool calls
Agent Memory: read-write via tool calls
Here’s what makes agent memory powerful:
The agent can now “remember” things, like user preferences, past conversations, and important dates. All stored and retrievable for future interactions.
This unlocks something bigger: continual learning.
Instead of being frozen at training time, agents can now accumulate knowledge from every interaction. They improve over time without retraining.
Memory is the bridge between static models and truly adaptive AI systems.
But it’s not as easy as it sounds.
Memory introduces new challenges RAG never had: memory corruption, deciding what to forget, and managing multiple memory types (procedural, episodic, and semantic).
Solving these problems from scratch is hard. If you want to give your agents human-like memory, check out Graphiti, an open-source framework for building real-time knowledge graphs.
You can find the GitHub repo here →
Good day!