[Tien] Birth defect's experiments results

 

 

 

https://www.medrxiv.org/content/10.1101/2025.04.05.25325217v1?

40 selected studies, 20 000 pregnancies, heart defect:
- AI: 95% - 96% accuracy, traditional diagnostic: 88 – 90% accuracy. On selected study data set

Sounds good but:

https://www.medrxiv.org/content/10.1101/2025.07.04.25330464v1.full
- “95% accuracy” misleading:
+ Most studies only identify: “Normal” or “Abnormal” (Binary classification) without specific diagnoses (13/20), (20/20) is limited to single organ, 1 study is multi-class classification – this one is commercial model, details on architecture is not available.

+ 3/20 models evaluated based on expertise sonographer, others relied solely on statistical performance.

+ most used limited gestational-age ranges

https://www.sciencedirect.com/science/article/pii/S2589537025001828?ref=pdf_download&fr=RR-2&rr=9f8be55d194aaa5f

+ high data set heterogeneity

+ small, non-blinded clinician comparison

-Difficulties in real-life deployment:
https://www.reuters.com/investigations/ai-enters-operating-room-reports-arise-botched-surgeries-misidentified-body-2026-02-09/?

+ AI mislabel body parts