On a phone call, the caller feels the gap between when they stop talking and when the agent answers. Buffered transcription waits for silence, then transcribes the whole utterance. Streaming transcribes small chunks while the caller is still talking. This toy model shows when streaming wins, and when the server can't keep up and falls behind.
Twilio Media Streams deliver 20 ms frames of 8 kHz audio. Streaming groups them into chunks. Each chunk is transcribed together with some earlier audio as context, because Whisper is trained on 30-second windows and gets less accurate on tiny clips. A request costs a fixed overhead plus compute for each second of audio it reads. If a chunk takes longer to transcribe than the chunk lasts, the next one waits and the delay grows for the rest of the call.
A real testbed would need: a websocket server for Twilio Media Streams, resampling to 16 kHz, a GPU or a hosted streaming speech API (CPU-only VPSs like the fleet's Contabo and Hostinger boxes will struggle with Large v3 Turbo at these chunk sizes), a merge rule for overlapping partial transcripts, and a recorded set of real calls to measure word error rate and response gap with and without streaming.