Anthropic
What OpenAI actually shipped
OpenAI's Decisions API is now generally available, and it is a narrower, stranger product than the usual model launch. There is no new chat surface here — just an endpoint, POST /v1/decisions, that takes your input plus a definition of the judgment you want and returns a machine-readable answer instead of a paragraph. Three output shapes are described: a probability, a single choice drawn from a fixed list you supply, and a rubric score.
The pricing is the part that will get quoted in every engineering channel: $0.10 per 1M input tokens, output tokens free, and a claim of roughly 10x the speed of the Responses API. Treat those as vendor claims until you have your own numbers — speed in particular depends enormously on input length, concurrency, and how many candidate labels you hand over.
Figure: Editorial context image from the source page (preview only; rights remain with the publisher) · Credit / further reading: dev.to
Why a judgment endpoint matters more than another model
Most production agents today contain a small embarrassment: a chat model asked to answer a yes/no question, followed by parsing code that hunts for "yes" inside a sentence beginning "Sure! Here's my assessment…". That pattern survives because it works often enough, not because it is good. It breaks on negation, on hedging, on multilingual input, and on the day the model decides to answer in bullet points.
A dedicated decision endpoint attacks that layer directly. Three consequences follow.
The parse layer disappears
If the response is already a probability or an enum member, the brittle string handling goes away. That is less code to maintain and, more importantly, fewer silent failures — the ones where a regex quietly returns false for three weeks and nobody notices because the fallback path happened to be plausible.
Latency budgets reopen
Agent loops are usually latency-bound, not intelligence-bound. A router that can be called ten times in the window one chat call used to occupy changes the design space: you can afford a cheap first-pass triage, a second-pass escalation check, and a third-pass confidence gate, all before the expensive generative call runs. The interesting architectures are the ones that decide first and generate second.
The label list becomes an interface
When the choice set is supplied by the caller, that list stops being a prompt detail and becomes an API contract. Versioning it, owning it, and monitoring its distribution is ordinary software discipline — applied to something that used to live inside a string.
The claim worth stress-testing
The 10x figure is the headline and the least portable number in the announcement. Speedups of that magnitude usually come from a shorter output path — no long generation, a smaller decode budget — so the real-world multiplier depends on how verbose your current prompts are. A tight one-line classification prompt will not see 10x. A three-paragraph "you are an expert triage analyst" preamble might.
The second thing to verify is calibration. A probability is only useful if it is honest. Before you gate anything on a 0.8 threshold, check what 0.8 has actually meant on your data, per language and per segment.
Figure: HEXOPS original illustration inspired by the topic (not a copy of the source article imagery) · Credit / further reading: dev.to
What builders should prototype
Pick one narrow, high-volume decision your system already makes badly — ticket urgency, refund eligibility, lead routing, order risk, moderation triage — and run it in shadow mode for a week. Log the current path's output and the Decisions output side by side against a labelled sample. Compare accuracy, latency at your concurrency, and cost per thousand decisions. Only then rewire.
Two design cautions. First, keep a fallback for low-confidence outputs; a probability with no threshold policy is just a number. Second, do not use a judgment endpoint where you need a justification string for compliance or customer support — the entire point is that it does not explain itself.
What to watch next
- Whether rate limits and concurrency ceilings hold up at production volume, which is where speed claims usually die.
- How rubric scoring is defined and whether rubrics themselves are versioned and auditable.
- Logging and data retention policies, which matter for cross-border operations touching customer data.
- Whether a distilled small model trained on your own decision logs undercuts $0.10/M at high volume. This is the classic pattern, and it usually arrives within a couple of quarters.
Key takeaways
- The Decisions API returns structured judgments — probability, enum, rubric score — rather than chat text.
- Reported pricing is $0.10 per 1M input tokens with free output tokens; the ~10x speed claim is a vendor figure to validate against your own prompts.
- The biggest win is architectural: it deletes the fragile parse layer sitting between the model and your control flow.
- Your label taxonomy becomes a versioned interface, which means it needs an owner, a changelog, and monitoring.
- Prototype in shadow mode on one narrow decision before touching production routing.
Next steps
- Export a week of real decision prompts from production logs and label a few hundred examples.
- Run a three-way comparison: current chat-plus-parse, the Decisions API, and a cheap local classifier.
- Define explicit confidence thresholds and a documented fallback path before any gating goes live.
- Write your label list down as a versioned schema, and treat every label change as a breaking change.
