# Make your voice AI answer before the silence gets awkward In a real conversation, people reply within about a fifth of a second. A voice AI that takes three seconds feels like talking to someone on a bad phone line: you start to wonder if it heard you, you start talking again, and you both collide. Most of that delay is not the AI thinking. It's waiting. Waiting to be sure you've stopped talking. Waiting to open a connection that closed while you were speaking. Waiting for a whole sentence before saying the first word of it. Each wait is a few hundred milliseconds, and they stack. Trillion runs on Deepgram for her ears, ElevenLabs for her voice and Claude for her brain. In one week we took her from about **3.0 seconds** to about **2.1 seconds**, measured from the end of your speech to the first sound of her reply. Here's where the time came from: | Change | What it saved | |---|---| | Ending your turn as soon as a finished sentence goes quiet | ~0.4s | | Speaking her first phrase instead of waiting for the full stop | ~0.3s | | A faster voice model (ElevenLabs Flash) | ~0.1s | | Keeping connections warm, and opening them while you talk | small, but every turn | | Caching the whole conversation | keeps long chats from slowing down | Two lessons surprised us, and they're built into the prompt: 1. **Your stopwatch probably lies.** We first timed turns from the moment the speech-to-text service reported your last words. Those words arrive about half a second after you said them, so every turn looked half a second faster than it felt. The prompt starts by building a stopwatch that can't flatter you. 2. **A smaller, "faster" model wasn't faster.** At this size, how long the AI takes to start answering depends mostly on the round trip to the provider, not the model. Measure before you switch. ## What you'll end up with - **An honest stopwatch:** a test that plays a recorded question into your agent, in real time, the way a microphone would. It times from the last sound of your voice to the first sound of its reply. It makes no sound itself. - **A per-turn breakdown,** so you can see exactly where each turn's time goes. - **The changes from the table above,** each one measured before and after on your own setup. - **Tests that make sure "faster" never means "cuts you off mid-thought".** ## How to use it 1. Open your voice agent's project in Claude Code, Cursor, or whatever coding agent you use. 2. Copy everything in the box below and paste it in. 3. Answer its questions. It measures first, then makes one change at a time and shows you the numbers before moving on. It's written around Deepgram and ElevenLabs, but every idea carries over to other speech services. It notes where to look if yours are different. ## The prompt ````text I have a voice AI agent: I talk, it transcribes me, a language model answers, and a text-to-speech voice speaks the answer. I want it to answer faster, measured from the moment I stop talking to the moment I hear the first sound of its reply. I want every change proven with numbers, and I never want "faster" to mean it cuts me off in the middle of a thought. BEFORE YOU WRITE ANY CODE, read my project and interview me. Answer what you can from the code yourself, then ask only what you couldn't find: 1. Speech-to-text: which service, and does it stream words while I talk (Deepgram, AssemblyAI) or transcribe after I stop (Whisper)? How does my code decide I've finished talking? 2. Text-to-speech: which service and which voice model? Does my code send it text one sentence at a time, or the whole reply? 3. The language model: which provider and model? Does my code stream the reply? 4. Connections: does my code create a new HTTP client for every request, or keep one? (Look. This one matters more than it sounds.) 5. Where does audio play: a browser page, a desktop app, a phone? Then summarize, in plain words, the path a single turn takes from my last word to its first sound, and wait for my go-ahead. WORK IN THIS ORDER. Measure first. Then make one change at a time, measure again with the same settings, and show me a before-and-after table. Keep a running table of every change and what it saved. If a change doesn't save time, say so and undo it. STEP 1: Build a stopwatch that can't flatter me. - Make a small test script that plays a recorded question into my agent the way my microphone would: the same audio format, sent in small chunks, in real time. (On a Mac, `say -o question.wav --data-format=LEI16@16000 "..."` records one.) It must make no sound itself: nothing is played out loud. - Start the clock at the last LOUD sample of the recording, as it's sent. Do not start it when the speech service reports my last words: those arrive late (often half a second), so every turn would look faster than it is. - Stop the clock at the first LOUD sample of the reply's audio, placed where it would actually be heard: when the first audio arrives, plus any silence the voice puts before the first word, plus any delay my player adds before playing. - Run 10 turns plus one warm-up turn, with about 8 seconds between turns (people pause, and idle connections close in that time). Report the median and the 90th percentile, the model, voices and services used, and whether the turns were warm or cold. - Also record, for every turn, the time of each step, all measured from when I stopped talking: a. the speech service decided I'd finished b. the words were final c. the model's first word arrived d. the first piece of text went to the voice e. the first audio arrived f. it was heard Store numbers only, never what was said. Show me the breakdown before changing anything. The biggest gap goes first. STEP 2: End my turn the moment a finished sentence goes quiet. Most agents wait for a fixed stretch of silence after the speech service's final words, and those words already arrive late, so the two waits stack (about 1.1s in our case). Change it to: - Fast path: when the speech service says it heard me pause (Deepgram sends `speech_final: true`; other services have an equivalent end-of-utterance signal) AND everything I've said so far reads as a finished sentence, end my turn once my voice itself has been quiet for about a quarter of a second, which by then it has. - "Finished" means it ends in . ? or ! (closing quotes are fine), is not an ellipsis ("Hmm..."), and doesn't end on a word that leads somewhere: and, but, or, so, because, if, then, that, the, a, to, of, for, with, um, uh, like. - Judge the WHOLE message, not just the latest piece: "I was thinking that maybe" plus "we could and" is not finished. - Anything else waits exactly as before. New words undo a "finished". An empty message from the speech service (its report of silence) does not. - Time the end of my speech from my voice's loudness, not from when words were reported, so the stopwatch in step 1 stays honest. Tests, before measuring: a finished question ends quickly; "set a timer for" doesn't; "...and." doesn't; "Hmm..." doesn't; my voice still going is never cut off; new words after a finished sentence undo it. STEP 3: Keep the connections warm. - Use one HTTP client per provider for the whole run, not a new one for every request. A new client means a new TCP connection and TLS handshake every turn. (If an SDK object is created per request, give it a shared HTTP client so the connections are reused.) - Keep idle connections open for about 60 seconds. Many HTTP libraries close them after 5 (httpx does), which is shorter than a pause between turns. - When listening starts, open the connections to the model and the voice service while I'm still talking: a HEAD request to each service's base address is enough. It needs no key and sends nothing of mine. - API keys are still read for every request, not held in memory longer. STEP 4: Speak the first phrase, not the first sentence. - For the FIRST piece of each reply only: send it to the voice at the first natural pause (a comma, colon, semicolon, or a dash with spaces around it) once it's at least about 25 characters long, if the rest of that sentence hasn't arrived yet. - Everything after the first piece is still sent in whole sentences, so the voice's rhythm doesn't suffer. - Never cut text that arrives already whole, like a fixed error message. - A comma inside a number ("8,849") is not a pause. Short openers like "Sure," or "Well," are not a phrase on their own. Tests: the first comma after 25+ characters cuts; "Sure, I can help." doesn't; "8,849 metres" doesn't; only the first piece is ever cut early; whole sentences fed in at once aren't cut. STEP 5: Try a faster voice model. - Measure time to the first audio byte, and the silence before the first word, for my current voice model against the provider's low-latency one (for ElevenLabs: eleven_flash_v2_5 against eleven_turbo_v2_5), six requests each on a warm connection. Show me both numbers and let me listen to both before switching: the faster one can sound slightly different. - Stream the audio as raw PCM if the provider offers it, so nothing waits on decoding. STEP 6: Cache the conversation, not just the instructions. - If my model provider supports prompt caching (Anthropic does), check what's cached. Often only the system prompt is, so every turn re-reads the whole conversation at full price and speed, and long chats get slower and costlier. - Mark the end of the newest user message as a cache point, and the one before it, so a turn with many tool calls can still find the previous cache. Stay within the provider's limit on cache points. - This only works if earlier messages are sent byte-for-byte the same every turn. If anything in the history is rewritten per turn (timestamps, injected context), move it into the newest message instead. - Show me the share of the prompt that was read from the cache, turn after turn. It should stay high instead of falling. STEP 7: Tell me where the floor is. When all of this is done, show me the final breakdown and be honest about what's left: how much is the speech service, how much is the model's round trip, and how much is the voice. Tell me which of those I can still change and which are the provider's. If I ask about a smaller model, measure it rather than assuming it's faster. GROUND RULES: - Measure before and after every change, with the same settings and the same gap between turns. A change that doesn't save time gets undone. - Never trade correctness for speed: cutting me off mid-thought is worse than being slow. - No new dependencies without telling me why first. - Nothing I say is logged or stored by the stopwatch, only numbers. - At the end, give me the before-and-after table and the one command that reruns the stopwatch. ```` ## What to check when it's done You don't need to read the code to know it worked: - **Ask a clear question,** like "What's the tallest mountain in the world?" The answer should start noticeably sooner. - **Say "Okay." and pause briefly** before continuing. If it jumps in, tell your agent, and limit the fast ending to longer sentences. - **Trail off:** say "I was thinking we could, um..." and wait. It should wait for you. - **Rerun the stopwatch** whenever you change models or services. Speed regressions are invisible until you measure. ## Why this works None of these changes make the AI think faster. They remove waiting: - **Waiting to be sure you've finished:** cut short when the words already sound finished. - **Waiting to reconnect:** gone, because the line stays open and gets opened while you talk. - **Waiting for a whole sentence:** gone, because the first phrase is enough to start speaking. - **Re-reading the whole conversation every turn:** gone, because it's cached. And the stopwatch keeps everyone honest: a speed-up you can't measure from the moment you stop talking to the moment you hear a reply isn't one.