Microsoft’s new model helps AI agents judge their next move
Microsoft released Microsoft-Decision-1, a Qwen3.5-9B-based model that scores agent decisions with calibrated probabilities.
Microsoft released Microsoft-Decision-1, a decision-scoring model post-trained from Qwen3.5-9B that returns a calibrated probability for each option in yes/no, multiple-choice, rating, and rubric tasks. Across 36 blind benchmarks covering nearly 150,000 questions, Microsoft said it was about 2.5 times faster than H2O-Lightning-4B and roughly 35 times faster than GPT-6 Sol in median latency. Internal Xbox, Copilot, incident-response, and Microsoft Discovery tests reported competitive quality at much lower latency and cost. It is generally available on Microsoft Foundry and OpenRouter at $0.042 per million input tokens, with no charge for output tokens.
- Microsoft-Decision-1 was post-trained from Qwen3.5-9B for single-pass scoring.
- It handles yes/no, multiple choice, ratings, and rubric grading, not chat.
- Microsoft reported median latency about 35 times faster than GPT-6 Sol.
- Available on Foundry and OpenRouter at $0.042 per million input tokens.
- Internal tests covered feedback sorting, Copilot grading, and agent replanning.
Full article497 words · extracted from helpnetsecurity.com · click to collapse
Microsoft-Decision-1 is a small decision-scoring model. Given a situation, a question and a fixed set of answer options, it returns a calibrated probability for each option.
“This model is designed for routing, classification, prioritization, verification, and workflow control, making it easier to incorporate decision intelligence into existing applications, agents, and workflows in a secure, trusted environment,” Achint Srivastava, VP of Software Engineering in Microsoft’s Office of the CTO, explained.
How Microsoft-Decision-1 works
Microsoft said it evaluated the model across 36 benchmarks covering nearly 150,000 questions, with the benchmarks kept blind from training. In the company’s benchmarking, Microsoft-Decision-1 was 2.5 times faster than H2O-Lightning-4B v1.1, the runner-up on speed, and about 35 times faster than GPT-6 Sol in median latency.
Accuracy, latency and calibration, side by side (Source: Microsoft)
The probability scores indicate the model’s confidence in each answer and can help an application decide whether to act, defer a decision or request a review. Microsoft advises developers to validate the model and confidence thresholds on examples representative of their intended use.
Microsoft-Decision-1 was post-trained from Qwen3.5-9B for single-pass scoring. The company plans to base future versions on other models, including those from Microsoft AI and OpenAI.
The model supports yes/no questions, multiple-choice questions, rating scales and rubric-based grading of AI responses and agent actions through structured API calls. It is not designed for open-ended text generation, conversation, translation or summarization.
The company identified speed, performance across tasks, robustness, probability calibration and safety as challenges in developing the model.
Internal testing
Microsoft is testing Microsoft-Decision-1 on tasks including sorting customer feedback, evaluating AI answers and finding relevant information during service incidents.
Xbox researchers used it to group more than 10,000 pieces of feedback and reviews from surveys, STEAM and X into predefined categories. Microsoft said the results were competitive in quality with GPT-6 Sol’s, with processing more than 14 times faster at one two-hundredth of the cost.
The Copilot team tested the model’s ability to assess chat and agent responses. Microsoft reported results competitive with GPT5.6 Luna’s, with processing 100 times faster. In incident-response tests, Microsoft-Decision-1 performed better and faster at retrieving relevant information than the large language model used for comparison.
Microsoft also tested the model in Microsoft Discovery, where AI agents evaluate experiment results and revise their next steps. The company reported that the model produced evaluation scores with 46 times greater consistency than the LLM-based scores and ran three times faster. The overall adaptive-replanning process ran nearly four times faster.
Other proposed uses include selecting an AI model to handle a request, checking content or code against defined criteria, ranking search results and deciding an AI agent’s next action.
Availability and pricing
Microsoft-Decision-1 is generally available through Microsoft Foundry, an Azure platform for building and running AI applications and agents. It is also available through OpenRouter, which provides access to multiple AI models through one API and account.
Pricing is US$0.042 per million input tokens, with no charge for output tokens.