In one developer’s account, rebuilding the orchestration behind a phone-based voice assistant reduced the wait for its first spoken audio from roughly nine seconds to about 1.5 seconds on a typical warm knowledge question. That is time-to-first-audio—not the time to finish the answer—and it is a case-study result, not an independently verified benchmark.
What the 1.5-second result measures
Mehar Aziz, a software engineer, reported the result in a DEV Community case study published September 11, 2026. The 1.5 seconds is the approximate time until a caller hears the beginning of a response on a warm knowledge turn. It does not measure the full spoken answer or apply to every kind of call.
As an Amazon Associate I earn from qualifying purchases.
The described scenario assumes retrieval is cached or already available, the language model streams its output, and text-to-speech begins once the first complete sentence is ready. Aziz did not provide a controlled test protocol, sample size, latency percentiles, or independent validation, so the figures should be read as measurements from one implementation.
The initial pipeline, as described by Aziz, waited for a final transcript, created a query embedding, searched a vector database, waited for the complete model response, sent that response to speech generation, and only then began playback. Aziz said this could leave a caller waiting roughly nine seconds on a typical company-knowledge question. After redesigning the flow, he reported about 10–30 milliseconds for on-box query embedding and about 1.5 seconds to first audio on the warm knowledge path. The embedding timing is one component, not a complete latency budget.
#1 Best Overall
- Stay present in every scenario: Every conversation is covered, in person, on calls, and online. 4 MEMS + 1 VPU microphones with AI beamforming capture every voice across the room. Smart Dual-Mode Recording switches automatically between phone calls and in-person. The free Plaud Desktop captures online meetings without a bot
- Walk out of every meeting with notes ready to act on: Plaud Intelligence transcribes in 112 languages with speaker labels and turns each recording into action items, decisions, and follow-ups, structured and ready to use. Choose from 10,000+ customizable templates tailored to your role and industry
- AI summary ready before you reach your desk: Auto Transfer moves each recording to the Plaud app automatically, and AutoFlow transcribes and summarizes so your notes are ready before you are back at your desk. Upgrade anytime to Pro (1,200 min/mo) or Unlimited
- Access your AI workspace anywhere: One connected workspace across Plaud Desktop, Plaud Web, and the Plaud mobile app, so your conversations and finished work follow you everywhere
- Your conversations stay private and yours: Compliant with ISO 27001, ISO 27701, SOC 2, HIPAA, GDPR, and EN 18031, with zero data used to train AI models. Trusted by 2.5M+ professionals, including legal, medical, and business professionals handling sensitive information
How the in-house voice system is arranged
The project began as an integration with a managed voice platform for automated onboarding calls. Customization requests, particularly for a more expressive voice, led Aziz to integrate speech generation directly and then take ownership of more of the orchestration. In the implementation described, Twilio handles telephony, Deepgram handles speech-to-text, Cartesia handles text-to-speech and voice generation, and an in-house server coordinates the call with a language model.
- Receive and stream the call. Twilio receives the incoming call and streams audio in both directions to the voice server over WebSockets.
- Transcribe caller audio. The server forwards caller audio for speech recognition and receives a transcript.
- Choose the appropriate path. The server routes the transcript by intent, deciding whether it needs a conversational response, company knowledge, or a backend tool.
- Retrieve only when needed. For a knowledge question, the system can search company-specific context. The case study describes document chunks and embeddings associated with an assistant, stored in Postgres with pgvector.
- Generate and speak incrementally. The server provides relevant context to the language model, streams generated text, and sends ready sentences to Cartesia. Speech audio returns through the call path so the caller can hear the response before the full answer is complete.
Document parsing, chunking, embedding, and storage happen outside the live call. That keeps the real-time path focused on query embedding, vector search, a small context block, and response generation rather than preparing the knowledge base during a conversation.
What changed to reduce the wait
Route before searching
Retrieval is a branch in this design, not an automatic step for every utterance. A fast router can separate small talk, knowledge questions, and tool calls. A greeting or acknowledgement need not trigger a company-document search; an appointment request may need a scheduling action rather than retrieval.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesRank #2
- Stay present in every scenario: Every conversation is covered, in person, on calls, and online. 4 MEMS + 1 VPU microphones with AI beamforming capture every voice across the room. Smart Dual-Mode Recording switches automatically between phone calls and in-person. The free Plaud Desktop captures online meetings without a bot
- Walk out of every meeting with notes ready to act on: Plaud Intelligence transcribes in 112 languages with speaker labels and turns each recording into action items, decisions, and follow-ups, structured and ready to use. Choose from 10,000+ customizable templates tailored to your role and industry
- AI summary ready before you reach your desk: Auto Transfer moves each recording to the Plaud app automatically, and AutoFlow transcribes and summarizes so your notes are ready before you are back at your desk. Upgrade anytime to Pro (1,200 min/mo) or Unlimited
- Access your AI workspace anywhere: One connected workspace across Plaud Desktop, Plaud Web, and the Plaud mobile app, so your conversations and finished work follow you everywhere
- Your conversations stay private and yours: Compliant with ISO 27001, ISO 27701, SOC 2, HIPAA, GDPR, and EN 18031, with zero data used to train AI models. Trusted by 2.5M+ professionals, including legal, medical, and business professionals handling sensitive information
Stream sentences to speech
Instead of waiting for the model to finish the entire answer, the system streams its output and buffers text until a complete sentence is ready. That sentence can go to speech generation while the model continues producing the rest. The design lowers the wait to first audio, but it also means the system may begin speaking before it knows every detail that will appear later.
Reuse connections
The implementation reuses language-model sessions, keeps a persistent Cartesia WebSocket for a call, and maintains warm sockets. The aim is to avoid paying connection setup time at a moment when the caller is waiting, such as immediately after a greeting.
Start some work before turn completion
Partial transcripts can be used to create a query embedding speculatively before end-of-turn confirmation. Aziz also describes removing filler words before caching and moving query embedding on-box. Speculation can save time when the caller’s utterance is correctly understood, but it requires the system to handle changed or incomplete transcripts safely.
Rank #3
- YOUR AI PERSONAL ASSISTANT FOR EVERYDAY PRODUCTIVITY: More than a voice recorder, Pocket works as your AI personal assistant to capture, transcribe, and summarize meetings, calls, and ideas instantly. Core features are included out of the box, with optional advanced tools available for power users.
- ONE-TAP RECORDING FOR REAL-LIFE MOMENTS: Capture meetings, phone calls, and in-person conversations instantly with a simple tap, no typing, no interruptions, just effortless note-taking anywhere you go.
- SMART AI INSIGHTS & ORGANIZATION: Pocket automatically turns recordings into clear summaries, key action items and structured conversation maps so you can quickly review what matters without digging through audio.
- TURN CONVERSATIONS INTO ACTION WITH “ASK POCKET”: Don’t just record, understand. Instantly ask questions across your meetings, extract key insights and generate next steps in seconds. All grounded in your recordings, so answers stay accurate and reliable.
- MAGSAFE COMPATIBLE FOR SEAMLESS USE: Easily attach Pocket to your iPhone or other MagSafe compatible devices for convenient, hands-free recording on the go. Perfect for capturing meetings, calls, and ideas without needing to hold your device.
Keep retrieved context small and trustworthy
The system uses a similarity threshold and concise retrieved context. If retrieval is weak, the safer response may be to acknowledge that the detail is unavailable or transfer the caller to a person rather than force an uncertain answer into the conversation. The case study also describes using a faster model where appropriate.
Manage the conversation, not just the model call
A low first-audio delay depends on the call loop around the model as well as on generation. The described system handles end-of-turn detection, eager turn handling, interruption cancellation, backchannels versus barge-ins, tool calls, transfers, and brief filler speech while backend actions run.
Why the result does not apply to every call
A warm knowledge question such as “What’s the TB test process?” can use already-available retrieval and begin speaking as soon as the first sentence is ready. A cold appointment booking may need to check availability, confirm details, call backend services, and produce options. Those are different workloads, and Aziz explicitly distinguishes such tool-dependent turns from the reported warm-path figure.
Rank #4
- Plaud Intelligence: Capture conversations in 112 languages and generate accurate transcripts with the Plaud App and Web. Plaud Intelligence uses leading models like GPT-5.5, Claude Sonnet 4.6, and Gemini 3.1 Pro to transform raw audio into structured insights. Choose from over 10,000 professional templates to generate mind maps and to-do lists, turning hours of discussion into immediate clarity
- Multiple Ways To Wear With Included Accessories: Adapt Plaud NotePin S to any workflow instantly with four included accessories. Wear your device effortlessly as a necklace, wristband, clip, or pin. Plaud NotePin S features a dedicated physical record button for precise, tactile control. Stay professional and keep your intelligence within reach all day
- Enterprise-grade Privacy: Built to the highest standards with ISO 27001/27701, SOC 2, HIPAA, GDPR, and EN18031 compliance. Every conversation is secure and protected. It is the trusted choice for creative, medical, and business professionals handling sensitive info
- Multimodal Input & Multidimensional Summaries: Capture audio, type notes, add images, and press/tap to highlight for richer context with multimodal input. Press the record button to mark key moments in real time. Plaud transforms a single conversation into multiple perspectives, providing faster, clearer insights, and unifies these inputs to deliver role-specific summaries that reflect your intent and priorities
- Lightweight Power and Peace of Mind: Weighing only 0.61 oz, Plaud NotePin S delivers 20 hours of continuous recording and 40 days of standby time. Store up to 64GB of audio locally, ensuring you capture every insight even without an internet connection
Time-to-first-audio is useful for judging when a caller hears a response begin, but it does not establish when the response is complete. A system can sound faster while still taking longer to finish a complex answer. A fair comparison between implementations would need the same workload and should distinguish warm from cold turns, first audio from end-to-end completion, and successful answers from transfers or tool-dependent responses.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What teams take on when they own orchestration
An in-house layer offers control over routing, retrieval, voice behavior, and integration with internal tools. It also makes the team responsible for operational details that a managed service may otherwise absorb: turn detection, interruptions, transfers, guardrails, connection management, recordings, transcripts, and failure handling across call edge cases.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
- Streaming risk: speech may begin before the answer is fully formed, exposing a partial statement that needs qualification. Shorter responses and retrieval-confidence thresholds can reduce that risk.
- Embedding consistency: locally generated query embeddings need to remain aligned with the embeddings used when documents were ingested.
- Heuristic upkeep: query rewriting and filler-word removal can improve a specific workflow, but heuristics require maintenance and may not generalize to other callers or topics.
- More failure points to own: a custom system must coordinate telephony, transcription, model output, retrieval, speech, tools, and conversational state. Latency improvements do not remove the need to engineer and operate those pieces.
How to evaluate a managed or custom approach
The case study reports no quoted prices or total cost, so it does not establish that building in-house is cheaper. A team evaluating the trade-off should compare the same calls under both approaches and account for engineering and operating work as well as service charges.
| Evaluation dimension | What to measure or clarify |
|---|---|
| Customization | How much control is needed over routing, voice, retrieval, tools, and call behavior? |
| First-audio latency | How quickly does speech begin on the same representative warm and cold workloads? |
| Answer completion | How long until the full response is spoken, not merely started? |
| Cold starts | What happens when connections are not warm, retrieval is uncached, or a backend tool must run? |
| Integration and operations | Who builds and maintains turn handling, interruptions, transfers, recordings, transcripts, and guardrails? |
| Total cost | What are the service costs and the staffing and operating costs at the team’s actual call volume? |
Aziz’s account is useful as an implementation example of reducing avoidable serial waits through routing, streaming, connection reuse, and earlier work. Its 1.5-second figure is best treated as a target for a particular warm knowledge path—not a promise about all voice AI systems or a substitute for testing the calls a team actually needs to support.
Quick Recap
Read Mehar Aziz’s case study on DEV Community.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
