Idea Lab · Oct 21 2026 · AI idea demo
A cascade sends every request to a small, cheap model first and escalates to a large one when the small model isn't sure. The whole design hangs on one number, the confidence cutoff, and most teams pick it by feel. Spillway takes a labeled sample of past cases, shows how escalation rate, error, latency and cost move as the cutoff moves, and recommends the cheapest cutoff that stays inside your error budget. Below is a working version on 400 synthetic cases.
Example data · 400 synthetic cases, generated in the page · not real benchmark results
Assumed per-call costs: small model 20 s and $0.02 per 1,000; large model 150 s and $0.60 per 1,000. Both are placeholders you'd replace with your own.
| Cutoff | Escalated | Error | Mean latency | Cost / 1k |
|---|