After MCP, A2A, & AG-UI, there’s another agent protocol:
And this one is built for Agent harnesses:
UHP (Unified Harness Protocol) is a standardized interface for running agent tasks across any agent harness, regardless of which one you use.
HarnessRouter is its open-source server implementation.
Here’s how it works:
Your product backend sends a task + harness_id to HarnessRouter.
HarnessRouter routes it to whichever configured harness should run it.
The harness streams progress and artifacts back through HarnessRouter.
HarnessRouter returns the output to your product.
Three core components handle this:
Gateway: Responses API, harness config, your model keys.
Runner: Isolated workspaces, one event stream, turn checkpoints.
Built-in tools: MCP servers, browser, database, GitHub, media.
This means your agent infrastructure becomes fully interchangeable.
Swap Codex for Claude Code without touching your product code.
Run the same task across different harnesses and compare outputs.
Add new harnesses as they emerge without any rewrites.
Here’s the repo: github.com/HarnessRouter/harnessrouter
(don’t forget to star it ⭐)
A Full Walkthrough on Multimodal Jev
Jev can see now!
We recorded a full video showcasing how to use it:
Chapters:
00:00 - Intro
00:28 - Multimodal Jev demo
02:13 - How Jev works
03:49 - Jev with images
06:51 - Demo app walkthrough
10:21 - Using multiple images
11:16 - Parallel questions
12:32 - Caveats and danger zone headers
12:59 - Outro
If you’re new to JEV and want to understand how decision models work, we’ve written a detailed article about it.
A tricky LLM interview question:
You move your LLM to a faster GPU.
Profiling confirms its kernels now finish sooner, but the time between generated tokens barely changes.
The model, precision, and batch size are identical.
Where did the expected speedup go?
(answer below)
An LLM forward pass is not one giant GPU operation.
It launches a sequence of operations for attention, matmul, normalization, sampling, data movement, and other work.
Without CUDA Graphs, the CPU prepares and submits these operations individually.
Each launch adds only a small amount of overhead, but token-by-token decoding repeats a similar sequence for every generated token.
This creates a strange bottleneck.
Some kernels finish in only a few microseconds. The GPU can complete one kernel before the CPU has submitted the next one.
That leaves empty spaces in a profiler trace. These gaps are not model computation. They show the GPU waiting for its next operation.
CUDA Graphs change how this work reaches the GPU.
The runtime captures a repeatable sequence of operations once, including their dependencies. Later iterations replay the captured sequence through one graph launch instead of submitting every operation separately.
To be clear, the kernels themselves do not become substantially faster.
Also, the model does not get smaller, and the forward pass performs almost the same computation.
Most of the gain comes from removing CPU launch work between kernels.
This is especially useful during LLM decoding.
Each decoding step processes one new token per sequence. This produces many short, repeated GPU operations where launch overhead can become a meaningful part of total latency.
Prefill often behaves differently.
A large prompt gives the GPU more parallel work, producing longer-running kernels. When those kernels already keep the GPU busy, reducing launch overhead matters less.
There is also a tradeoff.
A captured graph expects the same kernel sequence, stable arguments, fixed memory addresses, and a compatible execution structure on every replay.
Inference servers handle varying batch sizes and different prefill, mixed, and decode workloads. One captured graph cannot cover every execution path efficiently.
Engines such as vLLM and TensorRT-LLM therefore capture graphs for several common execution cases and batch sizes.
When a live batch does not match a captured size, the engine may use eager execution or pad the batch to a captured size.
Capturing more sizes improves graph coverage, but it also consumes more GPU memory and increases startup time.
So CUDA Graphs are not a universal inference speed switch.
They trade flexibility and setup cost for lower steady-state launch overhead.
The visual below shows the core mechanism.
Without a graph, the CPU submits each kernel independently and may leave idle gaps on the GPU timeline.
With a graph, the sequence is captured once and replayed as a unit.
CUDA Graphs are one example of why LLM inference needs a specialized serving engine rather than a standard prediction API.
Good day!
The padding-versus-eager choice is the part I'd want measured, not just described. Padding a decode batch up to the next captured graph size keeps the single launch, but it also spends GPU time on slots that aren't real requests, and that cost shows up as queueing delay for the next real batch. I'd want the serving trace to record which path ran and the captured size versus the actual batch, so a latency regression can be attributed to graph misses rather than to the model.
Once you can swap Codex for Claude Code behind the same harness_id, the eval suite has to sit outside any one harness. Otherwise a green run on A and a red run on B just means you measured two runners, not the same task. I'd pin the turn-checkpoint schema and score the streamed events the same way no matter which harness produced them.