A Server Unit runs the whole voice stack for a customer: call handling, recognition, the language model and speech synthesis. Each unit carries 100 simultaneous calls. Before that number goes into a contract it should be tested, so we built a load test and ran it twice.
We wanted to know whether the unit carries its rated 100 calls with replies that still sound like a conversation, and how it behaves past them, at 120%, 150% and 176%.
Measured on one Server Unit with a recruiting agent and simulated callers on the same unit, over G.711 A-law through a measuring bridge: 22 load steps in 9 series, 22,018 replies, about 129 hours of conversation. Load is given as a share of rated capacity (100 simultaneous calls) and reply time as an index, where a single call is 1.00.
In short: mean reply time stays in the Standard grade or better up to rated capacity, and to about 117% with early start. At 176% every call connected and none dropped. More than 99% of caller turns got a reply at every step, and a round of optimization for high load doubled the load the unit carries in the Standard grade.
How we measured
- Simulated callers on the same unit. Every conversation is two phone calls: the agent under test and a simulated caller. Twenty simulated callers, each with its own voice, manner and mood, ask about the job, pause and answer as people do. The Server Unit carries both ends of every conversation, so the telephony load is double what a real conversation of the same size would put on it.
- Realistic dependencies. The external systems the agent calls answer after a fixed, realistic delay.
- One clock on the line. A measuring bridge sits in the audio path. It marks the instant the caller’s phrase ends and the instant the first sound of the agent’s reply arrives. Reply time is that interval. It covers everything inside the unit, including the end-of-turn pause and the audio transmission.
- Steps. Load goes up in steps from one conversation to 176% of rated capacity. Each step settles for two minutes and is then measured for at least five. The single-conversation step runs for twenty.
- A step has to earn its place. A step counts only if it passes eight checks: steady state (the second half is not slower than the first), failed turns, health of the measuring bridge, overlapping speech, matching of bridge events to the agent’s log, dropped telemetry, call refusals and resource limiters.
- Two modes. We measured the agent in standard mode and with early start, in which the language model begins on the recognized text before the turn is formally closed.
What happened
Load bands
Grades describe how the reply feels to the caller. Each grade ends at a reply-time index.
| Load band | Load, % of rated capacity | Reply grade | Index up to | What the caller notices |
|---|---|---|---|---|
| Low | up to 42 | Platinum | 1.22 | Replies come back at close to single-call speed |
| Moderate | 42–65 | Gold | 1.49 | A brisk reply |
| High | 65–100 | Standard | 2.03 | A natural pause; this band ends at rated capacity |
| Peak | 100–137 | Minimum | 2.71 | A noticeable pause, still a conversation |
| Surge | 137–176 | Stretch | — | A longer pause, with every call still connected |
With early start each band extends further: Platinum to about 57% of rated capacity, Gold to about 78%, Standard to about 117%. We measured early start up to 128% of rated capacity.
Rated capacity is met
At 100 simultaneous calls the mean reply time is about twice the single-call value (index 2.02), and replies stay within the Standard grade. With early start the unit holds Standard to about 117% of rated capacity.
| Load, % of rated capacity | Reply index | With early start | 95th percentile / median | Throughput share |
|---|---|---|---|---|
| 1 call | 1.00 | 0.88 | 1.73 | 0.01 |
| 24 | 1.07 | — | 1.61 | 0.38 |
| 36 | 1.15 | — | 1.61 | 0.55 |
| 50 | 1.30 | 1.14 | 1.69 | 0.74 |
| 64 | 1.48 | — | 1.72 | 0.94 |
| 80 | 1.75 | — | 1.70 | 1.00 |
| 96 | 1.97 | 1.71 | 1.62 | 0.98 |
| 112 | 2.20 | — | 1.69 | 0.94 |
| 128 | 2.52 | 2.21 | 1.68 | 0.99 |
| 152 | 3.02 | — | 1.60 | 0.91 |
| 176 | 3.34 | — | 1.71 | 0.81 |
Reply time grows more slowly than load
We summarise the curve with one number, the load elasticity: how many percent the reply time grows for each percent of added load.
| Load range, % of rated capacity | Elasticity | Reply time per +10% load |
|---|---|---|
| 24–64 | 0.33 | about 3% |
| 64–128 | 0.77 | about 8% |
| 128–176 | 0.88 | about 9% |
The elasticity stays below 1 all the way to 176%, and the curve has no knee where it turns upward. A customer who runs 20% over rated capacity gets a measurably longer pause on a system that keeps working.
The tail stays in proportion
The median reply and the 95th-percentile reply keep the same ratio at every step, from one call to 176%:
The 95th-percentile reply is about 1.6 to 1.7 times the median at every load step.
Slow replies scale with the rest, so a customer who knows the typical pause can estimate the worst one.
Throughput holds
Replies per second climb with load, peak at about 80% of rated capacity and stay within 6% of that peak from 64% up to 128% of it. At 176% the unit still handles about 80% of its peak throughput.
Calls and media
- Turns answered. More than 99% of caller turns got a reply at every step.
- Media. The measuring line lost no RTP packets at any step.
- Admission. At 176% of rated capacity, 352 SIP calls were up at once. All of them connected, and none dropped.
- Models. The language-model queue and the speech-synthesis queue stayed empty at every step.
Round 1 and round 2
We ran the test, optimized the stack for high load and ran the test again. The optimization covered the media path, the way voice detection is scheduled across calls, the handling of silence in synthesized speech and the moment the language model is started.
| Measure | Round 1 | Round 2 |
|---|---|---|
| Standard-grade capacity | about half of rated capacity | rated capacity |
| Gold-grade capacity | about 36% of rated capacity | about 65% of rated capacity |
| Reply time, single call | baseline | 22% lower |
| Reply time at half of rated capacity | baseline | 34% lower |
| Peak throughput | baseline | almost double |
Round 2 reached the rated 100 calls, so the figure in the data sheet is a measured one. The Technologies page shows where the load test sits in the stack. The Voicecore post describes the next step in the audio path: a core written in Rust.
Test setup
Measured over G.711 on a telephony test line, with twenty simulated callers and a recruiting agent; 94% of first phrases came from the voice cache.