The 100M-Call Checklist: What Retell AI’s OpenAI Stack Teaches Engineering and Growth Teams
Retell AI reportedly uses OpenAI models to power 100 million monthly calls. GPT-6 Astra tops Retell’s voice-agent benchmark, and Codex helps the company reach a $1.6 million annual revenue run rate per employee. The headline is a model story. The operational lesson is a systems story: voice agents at that scale are won by evaluation harnesses, latency budgets, unit economics, and engineering throughput—not by a single leaderboard score.

Figure: HEXOPS original illustration inspired by the topic (not a copy of the source article imagery) · Credit / further reading: openai.com
How to use this checklist
This is a pass/fail gate review, not a maturity quiz. Engineering and growth teams should score each gate. Critical gates: E1–E4 and G1–G2. If any critical gate fails, do not scale call volume. If you pass 8 of 10 gates—including all critical gates—you are ready for a controlled expansion.
Engineering gates
E1. Benchmark on your own traffic, not the leaderboard
Public benchmarks tell you what a model can do on someone else’s prompts. Your customers call with accents, interruptions, background noise, and weird intents.
Pass if: You maintain a golden set of at least 500 real, consented/anonymized calls across your top three intents; every model or prompt change runs against it; GPT-6 Astra or your incumbent wins with a confidence interval you trust.
Fail if: You are switching models because a vendor chart looks good, or you cannot reproduce your own baseline.
E2. Hold a hard latency budget
Voice is less forgiving than chat. A brilliant answer that arrives late feels broken.
Pass if: You track p50 and p95 end-to-end response time, including speech-to-text, model, text-to-speech, and network; p95 stays inside your defined interactive budget; barge-in and interruption paths are tested.
Fail if: You measure only model latency, or p95 drifts without an alert.
E3. Know cost per resolved call
Call volume is a vanity metric if every call burns margin. Model spend must be tied to outcomes.
Pass if: You calculate cost per resolved call—model, telephony, infrastructure, and human escalation—weekly; gross margin is positive and improving; model routing sends simple intents to cheaper paths.
Fail if: You cannot answer “What does a resolved call cost?” or model spend grows faster than resolved calls.
E4. Run an automated evaluation harness
Retell’s benchmark claim is only meaningful if it reflects a repeatable test. Your team needs the same discipline.
Pass if: Every prompt, model, and tool change triggers regression tests for task completion, hallucination, latency, and safety; failures block deployment.
Fail if: QA is manual, ad hoc, or only happens after customer complaints.
E5. Design guardrails and handoff before scale
At 100 million calls, edge cases are not edge cases. They are daily traffic.
Pass if: The agent has clear escalation triggers, human handoff, PII redaction, abuse handling, and a kill switch; transcripts are audited on a sample basis.
Fail if: The agent loops, guesses on sensitive requests, or has no fallback when confidence is low.
E6. Use Codex as a force multiplier—with review discipline
The $1.6M revenue run rate per employee is not just a model achievement. It implies engineering leverage. Codex can help with test generation, migrations, dashboards, and integration glue.
Pass if: Codex-assisted work goes through the same code review, security checks, and tests as human-written code; you track PR cycle time and defect escape rate; repetitive work is automated first.
Fail if: Generated code bypasses review, or “AI productivity” is claimed without measuring delivery outcomes.

Figure: HEXOPS original illustration inspired by the topic (not a copy of the source article imagery) · Credit / further reading: openai.com
Growth gates
G1. Instrument the full call-to-revenue funnel
A voice agent is a growth channel only if you can see what happens after the call.
Pass if: Every call is linked to outcome, conversion, retention, and expansion where possible; dashboards separate containment, resolution, and revenue impact.
Fail if: You report call volume and average handle time but cannot connect calls to pipeline or churn.
G2. Optimize for resolution, not just containment
Containment can look great while customers leave angry. Resolution is the growth metric.
Pass if: You track resolution rate, repeat-call rate, CSAT, and escalation quality together; cost per resolution improves without harming satisfaction.
Fail if: You celebrate deflection while repeat contacts or cancellations rise.
G3. Run weekly experiments with controls
Model and prompt changes are product changes. They need experiments.
Pass if: You run controlled tests with pre-registered metrics, sample-size checks, and a weekly review; winners ship and losers are documented.
Fail if: Changes ship by opinion, or every experiment is a before/after with no control.
G4. Track revenue per employee and cost-to-serve together
The Retell headline—$1.6M annual revenue run rate per employee—is a useful operating metric, but only alongside margin.
Pass if: You report revenue per employee, gross margin, cost per resolved call, and quality in the same review; efficiency gains are reinvested into reliability or growth.
Fail if: You use revenue per employee as a vanity metric while ignoring support load, infrastructure cost, or quality regressions.
The scorecard
| Gate | Critical? | Pass signal |
|---|---|---|
| E1 Model benchmark | Yes | Golden set reproduces winner |
| E2 Latency budget | Yes | p95 inside interactive budget |
| E3 Cost per resolved call | Yes | Known, positive, improving |
| E4 Eval harness | Yes | Automated regression blocks bad deploys |
| E5 Guardrails | No | Handoff, PII, kill switch tested |
| E6 Codex leverage | No | Faster delivery with review discipline |
| G1 Funnel instrumentation | Yes | Call linked to outcome/revenue |
| G2 |



