Amazon Cuts Ticket Labeling Costs 98% With a Decision Model

A task that once ran on one of Anthropic's mid-tier models now runs on a far smaller specialist. Two engineers working on an internal ticketing system at Amazon replaced Claude Sonnet 4.6 with Strands Decider, a decision model from the Strands Agents project. They report matching quality at roughly 98% less cost, a saving they expect to reach millions of dollars in token spend over a year.

What a decision model does

A decision model is not a text generator. It receives a state and a question with a fixed list of options, and it returns a probability for each option along with a confidence score. There is no prose, no JSON to parse and no reasoning chain to pay for.

Ticket labeling fits that shape well. The ticket is the state, and the label catalog is the set of options. The tickets in question cover customer escalations, capacity requests, automated alarms and security findings, all of which have to land in one of 20 categories. Sonnet had been handling this with a long system prompt on every ticket, generating structured output against a fixed label set. That is a lot of tokens spent on a closed-choice problem.

How the swap was done

The team first tried the original 2B version of Strands Decider, then moved to a build based on Gemma 4 (the E4B variant), which scored clearly better on a test set. They tuned the confidence threshold needed before a label is applied, and on a separate set of tickets the E4B build beat the production Sonnet setup.

The serving stack is ordinary AWS plumbing:

• Ticket events land in an SQS queue, which triggers a Lambda function.
• The Lambda calls a SageMaker endpoint running the model on ml.g5.xlarge instances, each with a single A10G GPU.
• AppConfig holds a runtime switch, CloudWatch handles logs and alarms, and CDK defines the beta, gamma and prod stages.

A single request takes about 640 ms, and one instance can handle around 5,600 tickets an hour. At full utilization that would cost about $250 per million tickets. With headroom kept for traffic bursts, the engineers put the real figure near $500 per million, about 44 times cheaper than Sonnet.

Two things drove the token drop. The model emits a single choice instead of a JSON answer, and the prompt is much shorter. Production's label descriptions alone exceeded the model's 4,096-token window, so each of the 20 labels was rewritten as a one- or two-sentence criterion, with a "none" option added. On long tickets, the oldest comments are dropped to fit. Average Lambda execution time fell 74% on the day Sonnet was switched off.

The rollout pattern worth copying

The team avoided a blind cutover by using a three-way setting read from AppConfig on every message. In one mode only Sonnet labels tickets. In a dual mode both models run in parallel, Sonnet's result is applied, and the two outputs are compared. In the third mode only the decision model runs. Judging every disagreement on live traffic, they say, taught them more than any offline evaluation, and reverting is a 45-second configuration change.

They also hit a quirk: the order of JSON keys in a request shifted output probabilities by as much as 0.19. They now send requests with sorted keys, and anyone comparing runs against a decision model behind an API should check for the same effect.

Why it matters, and where it stops

The result is a reminder that large general-purpose models are often the default rather than the right tool. For high-volume work with a closed set of answers, a small model with a well-defined output can cut cost and latency sharply. It also fits a broader trend of running compact open-weight models, here built on Gemma, on modest GPU hardware.

The limits are real. The approach suits tasks where the options are known, volume is high and the useful context fits in a few thousand tokens. If labels need explanations, the catalog changes weekly, or the relevant context is large, a generative model still earns its price. These figures also come from the Strands team's own account and use published list prices, so independent replication will matter.

What to watch next is whether decision models spread beyond ticket triage into routing, moderation and tool selection inside agent systems, where a cheap, calibrated choice often beats a long generated answer.