LLM medical device regulation 🦆

Reading: If a therapy bot walks like a duck and talks like a duck then it is a medically regulated duck, npj Digital Medicine, 5 December 2025.

Medical device regulation depends on the claims and intended purpose of the device—amongst other things. In the case of Claude Sonnet, the system prompt suggests it does have an intended medical purpose:

Claude provides emotional support alongside accurate medical or psychological information or terminology where relevant.

That aside, the authors argue there's a case for pretty much all the LLMs to be treated as medical devices:

We argue that regulation needs to catch up with the reality of LLM deployment and use and apply the principle of ‘POSIWID’—the ‘purpose of a system is what it does'.

How can regulation catch up? By providing an open, dynamic, set of tests—better than nothing, and better than the "not for medical use" disclaimer. This could provide a lever for regulators to step in and take action for failing chatbots.