A demo is one call, in a quiet room, with a script the vendor chose. Your agent will take thousands of calls on noisy lines, from people who interrupt, answer in one word and change their minds. Ask these twelve questions before you sign. A vendor who can answer them with numbers, and say what each number was measured on, has done the work.
Each question below says why it matters, what a good answer sounds like and how we answer it. The plain list is at the end, ready to paste into an RFP.
How long does the agent take to reply, and what does that number include?
Why it matters. A late reply is the first thing a caller notices. Callers repeat themselves, talk over the reply or decide they are talking to a machine.
A good answer names a number, says where the clock starts and stops, and says whether the phone network is inside it.
Our answer. From the caller’s last word to the agent’s first sound, the agent itself needs about a quarter of a second: 232 ms to decide that the caller has finished, then 35 ms to the first sound. See Technologies.
What happens to that number under load?
Why it matters. Reply time on one call on an empty server says little about a busy Monday morning.
A good answer shows a load test: how many calls a server carries, how reply time changes as load rises, and what happens past rated capacity.
Our answer. A Server Unit carries 100 simultaneous calls. We load-tested it to 176% of that. Reply time grows more slowly than load all the way up, and at the top step every call connected and none was dropped. See Server Unit under load.
How does the agent know the caller has finished?
Why it matters. A silence timer either cuts people off in the middle of a number or makes every reply late.
A good answer describes a decision based on how the sentence ends as well as on silence, with figures from real calls.
Our answer. After a short pause, a model judges from intonation whether the thought is complete. On real calls it closed 87% of caller turns early and saved about 0.3 s per turn on average. See Knowing when the caller has finished.
What happens when the caller interrupts?
Why it matters. People talk over each other on the phone. An agent that cannot be interrupted feels like a recording; one that stops at every cough feels broken.
A good answer tells a real interruption apart from a cough, a keyboard or the agent’s own echo, and says what the agent does next.
Our answer. The agent tells an interruption from noise, dips its voice instead of stopping dead, and remembers what the caller did not hear. See Listening while talking.
How well does it hear my language, measured on what audio?
Why it matters. A figure on studio audio says little about a phone line, and a count of languages says nothing about how well each one is heard.
A good answer gives a quality figure per language, measured on phone-quality audio.
Our answer. Our recognition hears 48 languages, each graded in phone quality. 46 of the 48 are graded A+ or better. See Languages.
How does it handle one-word answers, names and numbers?
Why it matters. Most of what a caller says to an agent is short: “yes”, “Thursday”, a postcode. Short answers are where recognition fails most, and a wrong “yes” becomes a wrong booking.
A good answer has numbers for one- to three-word replies specifically, and a plan for names and digits.
Our answer. Oasis Speech has a path of its own for short replies, and it cut errors on one-word answers by 49%. The agent reads names, numbers and amounts back to the caller before it records them. See The shortest answers are the hardest.
Where do the models run, and does caller audio leave?
Why it matters. Every third-party AI service in the call path is another processor of your customers’ voices.
A good answer names where each model runs and offers a deployment that keeps everything inside your network.
Our answer. All models are ours to run: on our servers in the EU, in a regional cluster, or on your own servers, air-gapped if needed. No caller audio or text goes to a third-party AI provider. See Trust and data and Enterprise.
How is a change tested before a caller hears it?
Why it matters. A prompt edit that fixes one call can break ten others, and you want to know before your customers do.
A good answer describes simulated callers, a scoring method and a way to compare two versions of the agent.
Our answer. You can rehearse every agent in Polygon: simulated callers with their own facts, eight scored criteria, hard gates and a turn-by-turn comparison of two versions. See How we grade a conversation.
Can you grade my real calls, not only test calls?
Why it matters. Simulations are a start. Last month’s real calls are the test that counts.
A good answer is yes, with the same scoring for real and simulated conversations.
Our answer. Polygon imports real call transcripts and grades them with the same rubric and probes, so you can tell whether the new version does better on last month’s calls. See How we grade a conversation.
What may the agent do in my systems, and who decides?
Why it matters. An agent that writes to your CRM changes your records the way an employee would.
A good answer shows per-agent permissions, read and write scopes, and what happens before the agent acts.
Our answer. You declare tools per agent, you choose the read and write scopes of each connector, and a reply prepared ahead of time can only read: it calls no tool until the turn is confirmed. See Trust and data.
What happens to the price when call volume doubles?
Why it matters. Per-minute pricing turns a busy month into a budget problem.
A good answer gives a price you can know before the month starts.
Our answer. We charge a fixed price per agent, like a salary. The agent’s minutes are not billed; telephony is billed separately. See Pricing.
Can I talk to it before I sign?
Why it matters. A few minutes of talking to an agent show more of how it behaves than an hour of slides.
A good answer is a live agent you can talk to now, without a meeting.
Our answer. Talk to an agent in your browser by microphone, right now, with “Try it live” on our home page. Testing your own agent in the browser costs nothing.
Take the list with you
The twelve questions as plain text, for your RFP or your next vendor call:
- How long does the agent take to reply, and what does that number include?
- What happens to that number under load?
- How does the agent know the caller has finished?
- What happens when the caller interrupts?
- How well does it hear my language, measured on what audio?
- How does it handle one-word answers, names and numbers?
- Where do the models run, and does caller audio leave?
- How is a change tested before a caller hears it?
- Can you grade my real calls, not only test calls?
- What may the agent do in my systems, and who decides?
- What happens to the price when call volume doubles?
- Can I talk to it before I sign?