Microsoft AI beats doctors in medical trials

Microsoft’s AI diagnostic platform now holds a distinct advantage over human physicians in handling difficult medical cases. According to a study released Thursday in the journal Science, the system, known as MAI-DxO, correctly identified diagnoses in simulations at a rate more than double that of generalist doctors. Mustafa Suleyman, the CEO of Microsoft AI, framed these findings as a critical milestone in the pursuit of what the company calls medical superintelligence. This development positions Microsoft competitively alongside rivals like Google and OpenAI, which have recently made similar claims about their AI’s diagnostic capabilities.

The quantitative results highlight a sharp divide in performance. In tests using complex cases from the New England Journal of Medicine, the AI system achieved an accuracy rate of 83 percent. In comparison, primary care doctors and general internists, working individually with available reference materials, were correct only 36 percent of the time. It is important to note that such complex cases are typically managed by specialists or teams in real-world settings, groups that were not included in this specific comparison. The data suggests that much of the improvement stems from the power of underlying AI models, with Microsoft’s orchestration system adding incremental gains on top.

The study evaluated the system using models from multiple vendors, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest performance was recorded with OpenAI’s GPT-5 and GPT-5.5 models. For GPT-5.5, the orchestration layer raised accuracy from 72 percent to nearly 87 percent on complex cases, while slightly reducing costs. However, the benefits were less pronounced for emergency department cases, where both the standalone model and the orchestrated system scored around 85 percent at similar costs. Weaker models like Gemini 2.5 Flash saw a significant jump in accuracy from 46.4 percent to nearly 69 percent with the assistance of the diagnostic system.

Key figures in the effort are driving this strategy forward. Suleyman, who previously co-founded DeepMind, leads Microsoft’s frontier AI initiatives. He is joined by Dominic King, the vice president of Health at Microsoft AI, who emphasizes the growing gap between medical demand and supply. Suleyman noted that the company is collaborating with the Mayo Clinic to train a foundational healthcare model, expected for release next year. He predicts that by 2027, the industry will see major breakthroughs in processing medical records, allowing for accurate predictions of patient outcomes.

The financial and operational stakes are high for Microsoft. The company already sells dictation and radiology tools to clinicians and views this diagnostic capability as a natural expansion. Suleyman stated that the goal is to hold the top position in clinical decision support products in the coming years. He believes that as models approach theoretical maximums, the focus will shift to optimizing both accuracy and cost. The technology is now moving beyond benchmarking into real-world integration, with Microsoft aiming to deploy these systems across broader healthcare settings to improve overall outcomes.

To validate these capabilities in practical environments, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. This collaboration will evaluate the clinical reasoning of the diagnostic system across various hospital settings, including urgent care and emergency departments. The study design involves comparing the AI’s recommendations against those of human doctors who provide standard care to consenting patients. Dominic King described the initiative as potentially the largest of its kind, designed to provide compelling evidence for the large-scale deployment of AI in clinical workflows.