What is inside a Dialogist AI voice agent

Transcription takes 0.5 ms after the decision because recognition streams while the caller speaks and makes a trial pass during the pause. With early start the language model begins on that trial text inside the pause, so 3 ms of its work remain after the decision; in standard mode all 111 ms come after it. In the load test the early request left before the decision in 98% of replies, on average 225 ms ahead at 50 conversations and 508 ms ahead at 128. If the caller keeps talking, the agent drops the prepared reply and answers the whole phrase.

At 100 simultaneous calls on a Server Unit the same step takes about 0.4 s in standard mode and 0.25 s with early start. The Server Unit note has the full picture. Server Unit under load

• 89–235 ms saved

  Early start

  The language model starts on the recognized text while the agent is still waiting out the pause. The saving grows with load: 89 ms with one call, 189 ms at 96 conversations, 235 ms at 128.

• 280 → 40 ms

  Dead air removed from speech

  Synthesized sentences begin with silence; the median in our cache is 280 ms. The agent trims it to 40 ms as it plays the clip, and on the line a single call’s reply came about 200 ms sooner. The cache keeps the original.

• ≈ 0.3 s saved

  End-of-turn model

  After a 192 ms pause a small model judges from intonation whether the thought is finished. On real calls it closed 87% of caller turns early and waited out 96% of mid-thought pauses. It can run in observation mode first and only log what it would have done.

• 94%

  Voice cache

  The agent plays phrases it has said before from disk. The cache survives restarts, and out of hours the agent re-checks cached clips by recognizing them and regenerates any that fail.

• First sentence jumps the queue

  Under load, the opening of one call’s reply never waits behind the tail of another’s. The agent synthesizes the next sentences while the first one plays.

• One clock for all calls

  Media for every call runs on a single shared 20 ms tick, in place of a timer per call.

  Voicecore: Rust media backend
• 69 → 0.7 µs

  Cheaper audio frames

  Handing a frame between the telephony stack and the agent takes under a microsecond instead of tens.

  Voicecore: Rust media backend

Before and after: the same stand, a 20-minute single call and 50 conversations, recruiting agent. Cache: 64,192 phrases in the load test. Frame handoff: timed in isolation on the telephony library.

• Interruptions that mean something

  A caller can talk over the agent, and the agent stops. It detects an interruption from text or from sound. The acoustic mode has its own threshold and needs unbroken speech, so a keyboard, a slammed door or a cough does not cut the agent off. The agent ignores a short, low-confidence fragment over its own voice, which is what echo looks like.

• The voice steps back

  When the caller starts talking, the agent’s voice dips and dulls: lower volume, fewer high frequencies. If the caller says something that counts, the agent stops. After an “uh-huh” or a cough it comes back, from the nearest syllable boundary, the start of the sentence or the exact point. Set per agent.

• “Uh-huh” in the pauses

  Inside a long caller turn, the agent drops in short acknowledgements in its own voice. They are pre-rendered clips that a person has listened to and approved. They play in pauses, with a probability, a minimum gap, a cap per turn and no repeats, and never when the end-of-turn model thinks the caller has finished.

• It says when it is not sure

  Recognition confidence goes into the agent’s prompt. The agent sees a shaky transcription marked as such and asks the caller to repeat instead of guessing a name or a number.

• Silence handled politely

  A re-prompt, then a goodbye, timed from the true end of playback rather than from the end of synthesis.

• A room around the voice

  Optional office or meeting-room background, mixed under the voice from the first second of the call.

• The agent generates the greeting during the ring-out delay and streams it sentence by sentence.
• Wideband G.722 where the other side supports it. Smoothed gain control lifted signal-to-noise from 23 to 28.7 dB on 11 labelled real calls.
• The agent’s own “uh-huh” can echo back down the line from the caller’s handset. Recognition removes it from the transcript, and the agent keeps acknowledgements out of the conversation history.

Listening while talking: the full note

• Voice cloning

  A new voice from a 10 to 30 second sample and its transcript.

• Pronunciation dictionaries

  Brand names, streets and jargon said the way you say them, with a preview.

• One voice per agent

  From the catalogue or cloned, at the speed you set.

• Streaming synthesis

  Sentence by sentence: the next sentence is ready before the first one finishes playing.

RAGKnowledge Hub

Atlas is our knowledge hub. You connect an agent to one or several knowledge bases, and Atlas returns the passages that answer the question.

On-premise, the whole stack installs on your servers with Docker, offline if it has to be. A Server Unit carries 100 simultaneous calls and has been load-tested to 176% of that. Server Unit under load

A Server Unit carries the whole voice stack for up to 100 simultaneous calls. We stepped the load up from one call to 176% of rated capacity and watched reply time, the share of turns answered and the calls themselves. Then we optimized the stack for high load and ran the test again: Standard-grade capacity roughly doubled, and at the top step every call connected and none was dropped.

Measured on one Server Unit.

Server Unit under load

• ≈ 0.27 sthe agent’s own time from the caller’s last word to its first sound
• 35 msfrom the decision that the caller has finished to the first sound, with early start
• 96%of turns have the recognized text ready within 10 ms of that decision
• ≈ 100 msto the first sound of a freshly synthesized sentence (median)
• ≈ 1 msto play a phrase that is already in the voice cache
• 5–8×faster than real time: how quickly the voice is synthesized
• 94%of first phrases come from the voice cache
• 98%of replies start the language model before the pause ends
• > 99%of caller turns answered at 176% of rated capacity
• ≈ 150 msfor a fast knowledge search on the server

Recognition readiness: 96 to 97% of 58,000 turns across all loads. Voice speed: median 5.0 to 6.5× across load levels, cache misses only. First sound of a fresh sentence: median 76 to 106 ms across load levels.