In today’s newsletter:
Benchmark agent harnesses side by side.
Grok Bot masterclass: from 1 bot to a 14-bot org.
Benchmark agent harnesses side by side!
Most agent benchmarks compare models. But in production, the model runs inside a harness, and the harness changes cost, speed, and whether the task passes at all.
Harness Arena, inside HarnessRouter, benchmarks the harness and not just the model.
It runs one task across several harness × model configurations, and every config gets the same prompt, files, tools, skills, and evaluation checks.
For each config, it returns:
↳ Total cost.
↳ Execution time.
↳ Whether the task passed or failed.
↳ The files and artifacts it produced.
HarnessRouter supports Codex, Hermes, Claude Code, DeepSeek Harness, System One (powered by Jev), and 25 more.
GitHub repo: https://github.com/HarnessRouter/harnessrouter
(don’t forget to star it ⭐)
One thing we like about the design is that the cheapest run does not automatically win.
A config first has to pass the task. Only then does Arena compare cost and execution time.
In one published result:
↳ Hermes + GPT-5.2 passed 5/5 runs for 0.47 credits.
↳ Claude Code + Opus 4.8 passed 3/5 runs for 223 credits.
That is about 475x more credits for the config that also failed more often.
You can keep the model fixed and compare harnesses, or vary both. Either way, Arena answers practical questions:
↳ Should this task use Codex or Claude Code?
↳ Can a cheaper config still pass?
↳ Which setup works best for this workflow?
Once you pick a setup, you run it through the same UHP interface, with the same sessions, streaming, files, artifacts, cancellation, and failure handling.
So the comparison covers the whole runtime, not a model leaderboard.
Grok Bot masterclass: from 1 bot to a 14-bot org
We spent 100+ hours over two weeks building with Grok Bot. We now run 14 bots and talk to only 4 of them.
We recorded a 38-minute masterclass that covers all of it, from the first bot to a working org.
Here’s what the video covers.
00:00 - Intro
01:22 - What a Grok Bot really is (a new hire with a laptop)
02:23 - Every bot gets its own computer
03:35 - Installation and pricing
05:01 - Creating your first bot (no system prompt to write)
06:29 - Routines: tasks your bot runs on a schedule
07:30 - Computer use and connected apps
11:50 - Skills: a recipe card for your AI
13:07 - Teach a Task: turning a screen recording into a skill
18:38 - One fat bot vs. 14 bots (and why both fail)
19:39 - Run it like a company: you, executives, workers
21:00 - The Notion task board that tracks every bot
21:31 - Designing the org with Claude
23:11 - Why my bots are named after TV characters
23:50 - Leadership in one group chat
26:31 - Chief of staff: inbox sweep across Gmail, Slack and X
27:10 - Head of content: post analysis and competitor watch
29:44 - Design team: on-brand illustrations and Typefully drafts
32:46 - Engineering team: GitHub issue to PR with one label
35:17 - Claude Code on your bot’s computer
37:05 - Recap
38:12 - Outro
👉 Over to you: Which job would you hand to your first bot?
Good day!