Microsoft says it has built an AI system that diagnoses medical cases far more accurately than working physicians. The results appeared Thursday in Science, one of the field’s most selective journals. Microsoft AI chief Mustafa Suleyman told The Verge that healthcare AI is finally here.
The system is called Diagnostic Orchestrator. It runs several AI agents together rather than a single model. One agent reviews symptoms, another weighs medical history, and a third decides which tests to order. The group then converges on a diagnosis the way a hospital panel would.
The numbers behind the claim
Microsoft tested the system against human doctors on cases drawn from published medical reports. On several hundred New England Journal of Medicine cases, the AI reached roughly 83 percent accuracy. Generalist physicians using reference materials scored about 36 percent, and those working without references scored 22 percent.
Emergency department cases told a similar story. Using admissions data from Beth Israel Deaconess Medical Center in Boston, the AI scored 80 percent against 47 percent for human physicians. The system also tended to reach its answers at a lower estimated cost.
Microsoft built its own benchmark for the tests and calls it SDBench. The company plans to open-source it, which would let outside researchers check the results independently.
Built on GPT-5
The Orchestrator runs on OpenAI’s GPT-5. Microsoft says it also outperformed frontier models working alone, though the peer-reviewed paper stops short of newer releases. Models such as GPT-6 Astra, Google’s Gemini 4 Argon and Anthropic’s Opus 5.5 are not covered.
Suleyman has set a short timeline for the next milestone. He says Microsoft will build something indistinguishable from human performance within the next couple of years.
What the study does not show
Benchmarks are not clinics. The cases came from written reports and admission records, not from live patients in an emergency room. Real diagnosis involves incomplete histories, conflicting test results and time pressure that a curated dataset cannot reproduce.
Cost estimates inside a benchmark are also softer than real hospital economics. A system that orders fewer tests scores well on paper. Whether that holds when a missed diagnosis carries legal and human consequences is a separate question.
Accuracy figures also depend on how the comparison was set up. Doctors working without references scored 22 percent, an unusually low bar that few clinicians would recognise as their own practice.
Regulators are already moving. The FDA has let some AI medical tools reach patients before full approval. Authorities will now face a system that claims to outperform the clinicians it is meant to assist. That claim needs independent replication before any hospital acts on it.
Source: The Verge — Microsoft says its AI tool is better than doctors at diagnosing conditions
Related reading: FDA Lets AI Therapy Reach Patients Before Approval and FDA Clears First AI Sepsis Early Warning System.
The Netherlands, for internationals
One email a week: the week's news for internationals in the Netherlands, new practical guides and what changed in the cities. No spam, unsubscribe any time.
