Microsoft Decision-1 faces an independent test of cost and confidence

Choose the answers before asking the question

A support team might supply three categories—billing, technical problems and account access—and ask which fits an incoming message. Microsoft’s setup guide says the model returns a selected category and probabilities for each option. It produces no written explanation.

Microsoft says it built Decision-1 by further training the existing Qwen3.5-9B model. Developers can also request yes-or-no checks and ordered ratings, so an application can rank items or flag ones needing another look. The announcement below shows the approach.

An early use: sorting customer feedback

Microsoft reports that Xbox researchers used Decision-1 to classify more than 10,000 feedback and review items into themes they had defined. That is a concrete audience-research task: the researchers choose what to look for, and the model applies those labels across the material.

The company lists input pricing at $0.042 per million tokens—the small chunks of text a model processes—with no output-token charge. OpenRouter’s current listing matches that price and identifies Azure as its provider. A batch’s bill still depends on how much text and instruction each request contains.

Cheap decisions, with a measurable trade-off

Artificial Analysis’s new Decision Index combines four tests of knowledge, reasoning, tool selection and safety classification. Each contributes a quarter of the result. Microsoft-Decision-1 scores 54, level with Fastino’s GLiDE; TypeSafe’s Jev 1.13 leads the decision models at 58. Claude Haiku 5.5, using low reasoning effort, reaches 61. These are composite index points, not percentages of customer requests handled correctly.

The cost comparison gives Microsoft a stronger position. Its benchmark requests work out at $31 per million tasks, against $42 for Jev and $184 for Haiku. Those figures reflect the test’s mix of input lengths and current prices, rather than a flat fee for any million decisions. Haiku also takes a median 1.4 seconds per task, versus 0.13 seconds for Jev. Artificial Analysis introduces its findings below.

A probability is a promise worth checking

A confidence score of 0.9 should mean that answers assigned that score are right about nine times out of ten. Artificial Analysis finds that decision models generally fall short of their stated confidence on knowledge and reasoning questions. They are better calibrated on the tool-selection and safety-classification tests.

Microsoft leads the decision models on the knowledge test’s willingness to say “I don’t know”: it does so on 40% of questions it fails to answer correctly. That is an abstention measure, not 40% accuracy. Its actual accuracy on the same 500-question test is 28%. The table separates those measures from its stronger tool-selection result.

Before this evaluation appeared, Elvis, a DAIR.AI researcher, had welcomed Decision-1 while raising a broader concern from his work on decision models:

The new results give that concern a concrete test, without settling every use case. The suite includes advanced exam questions and safety labels, not Xbox’s customer-feedback categories. For a team sorting feedback, the useful next check is whether high-confidence labels are actually right on its own examples. A neatly printed probability still needs to earn its authority.

For European teams, check the deployment

The model guide lists Global Standard and Data Zone Standard in selected regions. Foundry’s deployment documentation explains the distinction: global inference may run across Azure regions, while a data-zone deployment confines it to that zone. The supported model and region must be checked together; creating a European resource alone does not establish where a global request is processed.