---
title: "Listening while talking"
description: "How a Dialogist agent keeps listening while it speaks, starts its reply inside the pause and tells an interruption from a cough."
canonical: "https://dialogist.cc/en/research/listening-while-talking/"
locale: "en"
updatedAt: "2026-10-24"
---

# Listening while talking, and what the agent does with the caller's open channel

Research · 2026-10-24 · Dialogist team

> How a Dialogist agent keeps listening while it speaks, starts its reply inside the pause and tells an interruption from a cough.

On a phone line both people can talk at once. A voice agent that answers only when the caller is silent and goes deaf while it speaks is half-duplex, and it sounds like a walkie-talkie. This note describes how a Dialogist agent keeps listening while it talks. It builds on an earlier note, [Knowing when the caller has finished](/en/research/knowing-when-the-caller-has-finished/).

The latency figures come from real phone calls (the end-of-turn model) and from the load-test stand with a recruiting agent (early start).

_The latency figures behind this note._

| Measure                                                                 |       Figure |
| ----------------------------------------------------------------------- | -----------: |
| Caller turns on real calls closed early by intonation, before the timer |          87% |
| Time given back per turn by those early closes, on average              |      ≈ 0.3 s |
| Replies that start the language model before the pause ends             |          98% |
| Reply time saved by early start, from one call to 128 conversations     | 89 to 235 ms |
| "Uh-huh" clips played to a caller without a person's approval           |            0 |

## What we mean by full duplex

The agent is a streaming pipeline, and the property we care about is that **the caller's channel stays open**. While the agent speaks:

- voice detection keeps running on the caller's audio;
- recognition keeps streaming it in 20 ms frames;
- the agent's own outgoing audio goes through a mixer that can lower, dull, pause and resume it, and mix other sounds in.

The sections below describe what the agent does with that open channel.

## Deciding that a turn is over

A silence timer chooses between two bad options: wait long enough to be safe and every reply starts late, or react to a short pause and cut people off mid-number.

The agent waits 192 ms, and then asks a small model whether the sentence sounded complete. If the model is confident, the turn closes at once. If not, the agent keeps listening, and a longer silence still closes the turn so nobody waits forever.

- The model's verdict applies on the next 32 ms frame of audio.
- On real calls (1,468 caller turns) the model closed 87% of turns early, 95% of them on its first look, and gave back about 0.3 s per turn on average. In 96% of the pauses where the caller was only thinking, it waited.
- **Observe first.** The model has three modes per agent: off, observe and commit. In observe mode it logs what it would have decided and changes nothing. We tune thresholds on real calls in observe mode, and switch to commit once the log agrees with what callers did.
- **Lists and numbers** are where we look hardest for early closes. A caller reading out a card number sounds finished between groups of digits.

## Starting the reply before the pause ends

Between the caller's last word and the closing of the turn there is a window of 200 to 400 ms. Recognition makes a trial pass during the pause, so the text is usually already known.

Early start puts that window to use. When the pause is long enough and the trial text is confident enough, the agent builds the prompt it would build after the turn closes, with the caller's expected reply on top, and sends it to the language model. The model writes the first sentence while the turn is still formally open.

The early request can only **read**, which makes it safe:

- it writes nothing to the conversation history;
- it does not call speech synthesis;
- it does not execute a tool, and it cannot end or transfer a call;
- if the turn closes with the same text, the same prompt and the same history, the agent takes the prepared sentences. If anything differs, the agent drops the request and the normal path runs;
- if the caller starts speaking again, or interrupts, the agent cancels the request and closes the HTTP stream.

In the load test the request left before the decision in 98% of replies, on average 225 ms ahead at 50 conversations and 508 ms ahead at 128. Early start cut the mean reply time by 89 ms with one conversation, by 189 ms at 96 and by 235 ms at 128.

> The saving grows with load: under load the turn takes longer to close, and the prepared reply has more time to be ready.

## Interruptions

Callers talk over agents. The agent decides two things in a few tens of milliseconds: whether this is an interruption at all, and whether it counts enough to stop.

### Two ways to detect one

A text-confirmed interruption needs recognition to have produced at least four letters of text. An acoustic one needs voice detection to hear sustained speech, which is faster. The acoustic mode has its own threshold, separate from the one that opens a normal turn, because a mistake costs more: a wrong "turn open" costs an empty buffer, while a wrong interruption cuts the agent off mid-word.

### What does not count

The acoustic mode demands unbroken speech above the threshold. A keyboard produces many short bursts; the counter resets at every gap between them and never reaches the limit. Speech holds it without gaps. A cough and a bare "uh-huh" do not count either.

### Echo

In browser calls an echo of the agent's own voice can reach the microphone. The agent recognizes it as echo, with no acoustic confirmation and low recognition confidence, and finishes its sentence.

### What the agent remembers

The agent marks an interrupted reply in the next prompt, so the model knows the caller did not hear all of it. Optionally the agent also receives the unspoken rest of the reply, and can pick it up or drop it.

### A switch

An agent can be set not to yield at all, for lines that must be read in full.

## Stepping back before stopping

Cutting off at the first sound is rude when the sound is "yes, yes". When the caller starts to speak, the agent's voice steps back first and stops only if the speech turns out to be an interruption.

- When the caller starts to speak, the volume ducks. By default it falls 15 dB over 150 ms and the voice also dulls: the high frequencies come down and a low-pass filter slides in, so the dip sounds like someone leaning back from the phone.
- In **hold** mode the agent pauses after about one syllable (200 ms). In **duck** mode it keeps talking more quietly.
- If the caller's speech counts as an interruption, the agent stops, with a 30 ms fade. If it does not, the voice returns over 300 ms.
- After a hold, the agent waits half a second in case the caller is about to speak again, then resumes from the nearest gap between syllables (up to 400 ms back), the start of the sentence or the exact point, whichever fits. If the pause was long (1.5 s) it goes back to the start of the sentence.
- If neither happens for 2.5 s, the agent decides whether to stop or restore its voice.

Each of these has a setting per agent, and an agent that is not allowed to be interrupted does not duck.

## Saying "uh-huh"

While a caller speaks for a long time, silence from the other end sounds like a dropped call. People fill it with small sounds.

The agent does the same, with a bank of short acknowledgements: "uh-huh", "right", "mm". The clips are **pre-rendered in the agent's own voice, and a person listens to them and approves them** before any of them can be used. If a voice has no approved bank, the feature turns itself off for that call, and the agent plays no unreviewed clip.

The agent plays one:

- inside a pause in the caller's speech (250 to 600 ms of silence), or on a timer;
- only after the caller has talked for 2.5 s;
- with a probability (0.6 by default), at most once per 3 s and 3 times per turn;
- never the same clip as one of the last four;
- not when the end-of-turn model puts the probability of "finished" above 0.4.

The agent leaves a played clip out of the conversation history, and when its echo comes back down the line from the caller's handset, the agent removes it from the transcript. All of this needs a pause window: the turn must stay open longer than the pauses the agent looks for. If the settings leave no room, the feature switches itself off for that call and the agent's card in Prompt Studio shows a warning.

## The rest of the room

- **Background.** Optional office or meeting-room sound, equalised, mixed in from the first second of the call. The agent does not count it as its own voice for silence timers and interruptions.
- **Silence.** A re-prompt and then a polite goodbye, timed from the end of playback on the line rather than the end of synthesis.
- **Confidence.** The agent passes the recognizer's uncertainty to the language model as a note on the turn, a number or marks on individual words. The prompt tells the model to ask again rather than guess a name or a digit.
- **Greeting.** The agent generates the greeting and streams it by sentence during the ring-out delay, so it starts the moment the line is answered.

## How we test it

- **Observe before commit** for the end-of-turn model, as above.
- **Prompt Studio** has a call tab: you ring the agent from the browser and hear the same greeting, silence handling and transfers as on the phone.
- **Polygon probes** check the behavior in simulated conversations ("did the agent ask again after a strange reply?"). [How we grade a conversation](/en/research/how-we-grade-a-conversation/) describes the method, and the [Technologies page](/en/technology/#polygon) describes Polygon.
- **The load test** checks that all of this holds with 100 calls running: see [One Server Unit under load](/en/research/server-unit-under-load/).

Tags: Research, Turn-taking

---

Contact: hello@dialogist.cc