Discussion about this post

User's avatar
Neural Foundry's avatar

The tool group scoping is clever, but I dunno if enough people realize the strip-markdown optimization is where the real savings come in. Most production agents I've worked with spend half their tokens on formatting artifacts that LLMs ignore anyway. Stripping markdown before it even hits the context window has cut our costs by like 45% in some workflows. It's one of those things that seems obvious after you do it but almost nobody thinks about upfront.

Expand full comment

No posts

Ready for more?