Actors for Agents: A Production Playbook for AI Workflows That Don't Fall Over

Most AI agent demos are one script, one process, one happy path. They call a model, call a tool, print something clever, and exit. Then a real customer shows up: the payment provider times out, the process restarts mid-task, two requests hit the same shopping cart at once, and the agent cheerfully re-charges a card it already charged. The model was never the hard part. The runtime is.

The useful mental shift is to stop thinking of an agent as a function call and start thinking of it as a small distributed system. That is exactly the lens the actor model gives you: isolated units with their own state, communicating only by messages, supervised by someone responsible for what happens when they fail. This playbook turns that lens into concrete engineering steps you can apply to any agentic workflow — order triage, catalog enrichment, refund handling, localized support — without rewriting your stack from scratch.

Why agent demos don't survive contact with production

Figure: Editorial context image from the source page (preview only; rights remain with the publisher) · Credit / further reading: www.infoq.com

Four properties of real traffic break naive agent code:

1. Time. A single task can span minutes or hours across model calls, human approvals, and third-party APIs. Your process will be redeployed during that window.
2. Partial failure. Tools fail halfway. A warehouse reservation succeeds, a payment capture fails, and now the workflow is in a state nobody designed for.
3. Concurrency. Two operators, or two retries from the same operator, act on the same entity at the same moment.
4. Mutation. The plan itself changes as the agent learns. The execution graph is not fixed at request time; it grows and rewires itself.

In-memory state, implicit retries, and shared mutable objects are fine for a demo and fatal for a workflow that touches money or inventory. The actor model does not magically fix these — but it gives each one an obvious place to live.

Seven steps to an actor-shaped agent runtime

Step 1: Give every workflow an identity and an address

Pick the natural unit of concurrency. For a cross-border store, that is usually per-order, per-SKU, or per-customer-thread — not per-request. Each unit becomes an actor with a stable address derived from your business key (e.g. order/9f2c...). Requests do not run the workflow directly; they send a message to that address. If the actor exists, the message lands in its mailbox. If it does not, it is created on demand.

This single decision eliminates the classic double-processing bug: two concurrent webhook deliveries for the same order become two queued messages for one actor, processed in sequence, not two racing threads.

Step 2: Make the actor the owner of its state

Once an actor exists, nothing else writes its state directly. The workflow's intermediate variables — which steps completed, which tool calls returned what, what the agent plans next — live inside that actor. Other components read via query messages, never by reaching into a database table that everyone writes to.

This is where write-through persistence matters. Do not keep state in memory and flush "later". Persist on every meaningful transition, before acknowledging the step. That makes restart survivable: a new process rehydrates the actor from the last checkpoint and continues. The cost is a small write per transition; the payoff is that a deploy in the middle of a refund flow is a non-event instead of an incident.

Step 3: Give every side effect an identity

Separate the actor's internal bookkeeping from the external world. Every outbound effect — charge a card, reserve stock, send an email, create a shipping label — should be a message with a deterministic idempotency key derived from the business key plus the step. The receiving integration must deduplicate on that key.

This turns "at-least-once delivery" from a scary phrase into an operational default. Retries become safe by construction rather than by hope.

Step 4: Define the protocol, not just the prompt

Write down the message types the actor accepts: commands ("capture payment", "approve refund") and events ("payment captured", "stock reserved"). Version the schemas. Validate on receipt and reject unknown fields loudly rather than silently.

Teams routinely skip this and then discover that half their production incidents are really protocol drift — a new field added by one service, an old consumer ignoring it, and an agent confidently acting on a stale interpretation.

Step 5: Draw supervision boundaries before you need them

Supervision is the underrated half of the model. For each actor, define: who supervises it, what a child failure should do (restart, escalate to a human queue, or stop the workflow), and how many restarts are tolerated in a time window before it is parked.

Be explicit about what is not retryable. A malformed model response can be retried. A compliance rejection cannot. Encoding that distinction in supervisors — rather than in scattered try/except blocks — is the difference between a system that self-heals and one that quietly loops.

Step 6: Treat the execution graph as data

Agentic workflows are not linear. The agent decides to loop back, branch, or spawn subtasks. If that graph lives only in a prompt, you cannot resume it, audit it, or explain it.

Persist the graph: nodes are steps, edges are transitions, and each node carries its status and result. A continuation is then just "load the graph, find the pending nodes, resume". This is also what lets you put a human approval node in the middle and have the workflow wait days without holding a thread open.

Step 7: Bound everything

Mailboxes need caps. Tool calls need timeouts. Workflows need deadlines. Model calls need token and cost budgets per actor per hour. Anything unbounded becomes an outage with extra steps — one runaway retry loop can burn through a month's inference budget before a human notices.

Add a dead-letter path for messages that exhaust their retries, with enough context that an operator can act on it without reading logs.

Figure: HEXOPS original illustration inspired by the topic (not a copy of the source article imagery) · Credit / further reading: www.infoq.com

The parts that matter most for cross-border teams

If you ship to multiple regions, a few extra constraints fall out naturally from the actor shape:

• Pin the actor to a region. State ownership makes data residency easy to reason about: the actor's persisted state lives where the actor lives. Do not let a message route a EU order's state through a US-hosted writer.
• Keep currency and tax logic inside one owner. Multi-currency rounding and tax-inclusive pricing are common sources of double-charging across retries. One actor per order owning the computed totals is far safer than recomputing totals in three services.
• Expect the long tail on third-party latency. Cross-border payment and logistics providers have wide latency distributions. Timeouts and dead-letter queues are not optional polish; they are the normal operating mode.
• Localize at the edge, decide at the actor. Language and formatting belong to presentation. Do not let a locale-specific prompt variant become a different business rule.

Pre-launch checklist

• Every workflow has a stable business-key address, and inbound requests are messages, not direct calls.
• State is written through on each transition; no in-memory-only progress.
• Every external side effect carries a deterministic idempotency key, and receivers deduplicate.
• Message schemas are versioned and validated on receipt.
• Each actor has a named supervisor, a defined failure policy, and a restart budget.
• The execution graph is persisted and resumable without an open thread.
• Mailboxes, tool calls, workflows, and model spend all have explicit limits.
• Dead-letter queues are monitored by a human, not just a dashboard.
• Every message carries a trace ID so a single workflow can be replayed end to end.
• You have run a "kill the process mid-workflow" drill in staging and watched it resume.

Key takeaways

• The model is not your reliability problem; the runtime is. Split the two.
• One actor per business entity turns concurrency bugs into a queue.
• Write-through state is what makes deploys, crashes, and long human waits survivable.
• Idempotency keys on every side effect make retries the default, not the risk.
• Supervision boundaries encode the difference between retryable and terminal failures.
• Persisting the execution graph is what turns a clever prompt into an auditable process.

Next steps

Start with your highest-blast-radius workflow — refunds, payment capture, or inventory reservation — and run steps 1 through 3 this sprint. You do not need an actor framework to begin; a queue plus a persisted row plus an idempotency key gets you most of the value. Then add supervision and graph persistence before you scale the number of concurrent workflows, not after. Finally, schedule the failure drill. A workflow you have never killed on purpose is a workflow you do not yet understand.

Sources

• InfoQ presentation by Manju Rajashekhar, Actors for Agents: Patterns for Production AI: https://www.infoq.com/presentations/actors-for-production-ai-agents/