The U.S. Food and Drug Administration's Digital Health Center of Excellence has released a discussion paper exploring how generative AI-enabled medical devices should be evaluated and regulated. The agency is seeking public feedback on approaches covering both premarket assessment and post-market monitoring, acknowledging that generative AI introduces risks and behaviors that may not be adequately addressed by traditional medical software evaluation methods.
Unlike conventional software with relatively predictable inputs and outputs, generative AI systems can accept open-ended requests, perform multiple tasks and generate different responses to similar inputs. Their performance can also change as developers modify underlying foundation models, prompts, retrieval methods, guardrails, orchestration logic or interfaces. The FDA highlighted potential risks including convincing hallucinations, unexpected behavior and performance degradation after deployment.
The agency is considering a risk framework based on two main factors: the level of autonomy given to the AI system and the potential clinical consequences if its output is incorrect. For premarket evaluation, the FDA is exploring a competency-based approach similar to how clinicians are assessed. Depending on the intended use and risk level, this could combine standardized benchmarking with clinical validation through retrospective patient data, simulated patient interactions, independent clinician review or testing in real clinical environments without allowing AI outputs to influence patient care.
The FDA is also considering whether certain generative AI devices should be evaluated against qualified clinicians performing comparable tasks. After deployment, manufacturers could be expected to periodically retest systems, monitor changes in performance and have clinicians review samples of real-world outputs. The agency is additionally exploring mechanisms that would allow manufacturers to predefine certain model updates, potentially avoiding the need for a completely new regulatory review after every modification.
Another proposal involves voluntary Foundation Model Device Master Files. AI developers could confidentially provide the FDA with information about foundation models—including training methods, known limitations, safety measures and healthcare-specific evaluations—which medical device manufacturers could then reference in their own regulatory submissions. This could simplify reviews for multiple products built on the same underlying AI model.
The discussion paper does not introduce new requirements or represent final FDA guidance. Instead, the agency is seeking industry input as it develops a flexible, risk-based regulatory framework capable of addressing the rapidly evolving nature of generative AI while maintaining appropriate safeguards for patients.
Click here to read the original news story.
