Gemini's Fresh Ideas playbook says a small model's own confidence score is a bad signal for deciding when to hand a task to a bigger model. This page tests that claim on example data. It runs the same 400 tasks through three routing rules for a kev-0.8b → kev-4b cascade and shows how many wrong answers slip through without escalation, and how many risky approvals get through.
All tasks, scores and outcomes below are synthetic example data generated in your browser. Nothing here was measured on the real Kev.
Learn how often the small model is actually right at each confidence level, using half the traces as training data.
Tasks touching money, credentials, deletion or the public edge always go to the bigger model or a human, whatever the score.
The playbook gives no formula for RSI or the "execution manifold", so there is nothing to implement. Calibrated accuracy plays this role here.
Preempting work across nodes in 15 ms is a scheduler problem for a large cluster. With one Contabo box running Kev it does not apply.
Each curve sweeps the threshold from 0 to 1. Lower and further left is better: fewer wrong answers kept by the small model for the same amount of escalation. The dots mark the threshold you set.
What the small model claims vs. how often it was right. A large gap in the top bins is the overconfidence problem the playbook describes.
About 200 real Kev decisions labeled right/wrong (the 116-case benchmark is a start), a binning or isotonic calibration step in the gateway, a stakes list kept as hard rules outside the model, and a weekly re-check of calibration as task mix shifts. That is a few days of work on the existing gateway, not a new system.