The confidence-threshold routing is the part worth stealing. In planning work it maps cleanly: cheap deterministic rules should handle the routine calls — which goal a surplus goes to, whether term cover clears the income multiple — while anything genuinely uncertain escalates to a human.
TrueForge holding every write for approval is doing a different job than Daytona keeping credentials outside the sandbox. The five wrong matches Opus caught were low-confidence Jev choices that would have gone through otherwise, so the approval has to be on the exact issue id and payload, not on the agent's summary of the match. A sandbox that cannot see the secret still runs a script the harness then signs.
Those 5 catches only cover matches that scored low. In a weekly agent the issue list keeps changing, so I'd also pull a small sample of high-confidence auto-accepts each week and check them by hand, since a confident wrong match skips review entirely.
The confidence-threshold routing is the part worth stealing. In planning work it maps cleanly: cheap deterministic rules should handle the routine calls — which goal a surplus goes to, whether term cover clears the income multiple — while anything genuinely uncertain escalates to a human.
TrueForge holding every write for approval is doing a different job than Daytona keeping credentials outside the sandbox. The five wrong matches Opus caught were low-confidence Jev choices that would have gone through otherwise, so the approval has to be on the exact issue id and payload, not on the agent's summary of the match. A sandbox that cannot see the secret still runs a script the harness then signs.
Those 5 catches only cover matches that scored low. In a weekly agent the issue list keeps changing, so I'd also pull a small sample of high-confidence auto-accepts each week and check them by hand, since a confident wrong match skips review entirely.