What DoorDash actually changed

Figure: Editorial context image from the source page (preview only; rights remain with the publisher) · Credit / further reading: www.infoq.com
DoorDash's grocery reorder flow used to take about 16 seconds end to end. According to the engineering work described by InfoQ, it now completes in roughly 2 seconds. The mechanism is not a faster model or a smaller one — it is a change in when the model runs and what is allowed to sit on the critical path.
Three moves did the work.
First, batch LLM inference precomputes each shopper's likely recurring needs ahead of the request, so the expensive reasoning happens off the clock rather than inside the click. Second, a deterministic validation layer checks the model's output before it reaches the user — a rules-and-schema pass, not a second model asked to grade the first. Third, live catalog resolution reconciles those precomputed selections against current products, prices and availability at render time, so a prediction made hours earlier does not surface an item that has since been delisted or repriced.
An editable shopping interface closes the loop. When the precomputed basket is wrong, the shopper corrects it directly instead of triggering another inference round.
The reported outcomes: shopping list views doubled, and the flow's order rate rose by roughly 34%. Both are company-reported figures without a published baseline definition, so treat the direction as the signal and the decimals as marketing.
Why 16 seconds is a product problem, not an infrastructure metric
Grocery reorder is the most habitual interaction in e-commerce. The basket is largely stable week to week — the same milk, the same brand, the same pack size. Sixteen seconds is long enough for a user to remember they were going to do something else, switch tabs, and not come back. Teams tend to file latency under SRE concerns; in a repeat-purchase flow it is a product surface with a revenue line attached.
The same stability that makes reorder valuable makes it a poor fit for live reasoning. Most of the intellectual work is identical to what the system already computed last week. Running a general-purpose model on every request to rediscover a known basket is expensive, slow, and nondeterministic — three properties you do not want on your highest-frequency conversion path.
The useful question for other teams: which of your AI features is secretly a batch job wearing a request-time costume?
The pattern: predict ahead, verify cheaply, resolve live
The architecture earns its keep by separating three concerns that are usually collapsed into one prompt.
Prediction can be stale, can be probabilistic, and can be recomputed on a schedule. Validation must be deterministic, cheap, and in the hot path. Resolution must be live, but it is a lookup, not a generation. Keeping the model in the first bucket and out of the other two is what turns an eight-second model call into a two-second page.
There is a cost story here too. Batch inference is cheaper per token and schedulable against idle capacity; live inference under a traffic spike is where both the bill and the p99 blow up simultaneously. Precomputation converts a variable cost that scales with clicks into a fixed cost that scales with users — a materially better curve for a flow people hit every week.

Figure: HEXOPS original illustration inspired by the topic (not a copy of the source article imagery) · Credit / further reading: www.infoq.com
The risk is staleness, and it does not fully disappear. Live catalog resolution covers SKU drift — discontinued items, price changes, stock-outs. It does not cover intent drift: a new household, a dietary change, a move, a baby. The editable interface is the honest admission that predictions fail and the user is the fallback plan. That is a healthier design than pretending the model is always right and hiding the correction behind a support ticket.
Where deterministic validation earns its keep
Guardrails in this flow are unglamorous: schema conformance, quantity sanity checks, allergen flags, price-delta thresholds, prohibited substitutions. Every one of them is auditable. You can explain to a merchant or a regulator why the system proposed a given basket. A model checking a model is neither cheap nor explainable, and it fails in correlated ways — when the first model is confidently wrong, the second one often agrees.
What "agents" means here
The label is doing less work than it appears. This is a pipeline with a model in one stage, bounded by validation and reconciliation — not an autonomous loop negotiating with a catalog. The gain came from constraining the agent, not enlarging it. That is the transferable lesson.
What to watch next
- The precompute window. How far ahead is the prediction made, how often is it refreshed, and what event triggers a re-run? Too long and you waste compute on users who will not order; too short and you are back to live inference.
- Whether the pattern spreads. Restaurant reorders, subscriptions and replenishment reminders all share the stable-basket property. If this works in grocery, the same shape should appear elsewhere in the product.
- Evaluation of wrong guesses. What is the precision and recall of "did we correctly predict the recurring item"? Unreported, and the most important number for anyone copying this.
- The 34%. Lift versus a real control group, and how much is attributable to latency alone versus the new editable interface. Those are different investment decisions.
- Unit economics. Batch inference is not free. The win is only real if total inference spend is flat or down, not merely redistributed.
Key takeaways
- DoorDash reports grocery reorder latency dropping from ~16s to ~2s, with shopping list views doubling and order rate up ~34%.
- The gain came from moving LLM work off the request path via batch precomputation, not from a faster or smaller model.
- Deterministic validation — schemas, quantity sanity, allergen and price checks — sits in the hot path and is auditable in a way model-on-model checking is not.
- Live catalog resolution handles SKU and price drift; an editable interface handles intent drift.
- High-frequency, low-variance interactions are the best candidates for precomputation and the worst candidates for per-request reasoning.
- Faster is not automatically better: measure whether the extra accuracy of live inference is actually worth the seconds it costs.
Next steps
- Audit your own AI surfaces for request-time work that could be a scheduled job. Start with anything a user hits more than once a week.
- Split each feature into prediction, validation and resolution layers, and write down which layer is allowed to be nondeterministic.
- Add a deterministic post-filter in front of any model output that touches money, inventory or allergens.
- Instrument prediction accuracy separately from conversion, so a latency win never masks a relevance loss.
- Define a freshness budget per feature: how stale may a precomputed answer be before it must be recomputed or discarded?



