Case Study: What AI/ML Device Recalls Reveal About Emerging Risks
Description
Note: this is an updated version of the original article first published on Sep 8. It has been corrected for an error in the number of AI/ML enabled medical devices cleared by the FDA mentioned in the original article.
In this audio brief, we unpack recalls data on AI/ML enabled medical devices to gain insights on emerging vulnerabilities from a risk management point of view.
Here are a few key highlights
* The Landscape: 1,247 FDA cleared AI/ML devices across 155 product codes; 38 recall events identified for a deep dive.
* Recall Severity: Mostly Class II recalls, no Class I recalls.
* Leading Causes:
* Software and algorithm errors (e.g., incorrect dose calculations).
* Data integrity issues (e.g., misfiled or missing images).
* Hardware failures (e.g., loose CT table bolts).
* Labeling & approval lapses (e.g., unapproved software versions).Four-tier, risk-based system; stringent requirements for high-risk systems (including many medical devices); compliance timelines of 1–3 years.
* Trends to Watch:
* High rate of recalls within 12 months of clearance.
* Devices without clinical validation face more, and larger, recalls.
* Public companies account for nearly all recalled units, suggesting market pressures for faster launches without adequate clinical validation.
* Takeaways for stakeholders:
* Manufacturers: Strengthen lifecycle controls, prioritize pre-market validation, enhance post-market vigilance.
* Regulators: Consider time-limited approvals and stronger oversight of high-volume AI devices.
* Clinicians: Validate AI results with clinical judgment—trust but verify.
* Patients: Benefit from innovation but remain vulnerable; safety must remain paramount.
AI in MedTech is transformative but not without risk. The challenge is moving from compliance - driven recall response to active risk mitigation for robust safety and effectiveness.
🎧Listen to the audio brief above for an overview of the AI/ML device recalls, emerging vulnerabilities and trends to watch.
Thanks for reading Let's Talk Risk!. This post is public so feel free to share it.
Note:
This audio brief was prepared using Google NotebookLM, an AI-enabled research tool. Here is the list of resources used in our analysis:
* JAMA: Early Recalls and Clinical Validation Gaps in Artificial Intelligence - Enabled Medical Devices, Research Letter | AI in Health Policy, August 2025.
* AI/ML Recalls Analysis - Unpublished report, created using ChatGPT.
This summary was created using ChatGPT-5 (September 2025) with expert review. It distills publicly available information on FDA-cleared AI/ML-enabled devices and related recall patterns. While reviewed for accuracy and relevance, it does not constitute legal, regulatory, or medical advice. AI in healthcare is a rapidly evolving area, and details may change after publication.
We encourage listeners to interpret these findings in the context of these constraints.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit naveenagarwalphd.substack.com/subscribe