Who is responsible when AI makes a mistake?

Another thing from that MHRA session I mentioned is the thinking around mistakes.

What I took from Lawrence Tallon’s comments is that there are two elements: safety and then liability.

Safety

For drugs, there’s a chunky barrier to market because once you swallow it, you’re committed.

The difference for AI is that it’s not typically all or nothing. It might be a decision support tool, for example.

The MHRA emphasis is more on continual monitoring and reporting. Models do drift over time, or have different performance in different settings. I’d expect efficacy and safety work, but proportionate to the risk.

Liability

Thinking about harm to patients, the MHRA don’t know who liability lands with. Law and finance are having the same discussions.

For clinicians, their necks could be on the line if they do accept AI input, or if they don’t. The number of combinations of circumstances is worrying: model sensitivity, selectivity, applicability, competing evidence, mitigating factors… I’m sure the list goes on.

The position, if I’ve understood it, is that if a clinician is using an AI tool, making normal reasonable decisions, then they are practising as everyone would expect.

What we want to avoid is a situation where useful products don’t get built because of litigation fear.

* * *

There will be a report on this in the autumn, but litigation questions probably won’t be resolved by then. The phrases used were along the lines of “case law is evolving” and “we need a clear enough” understanding.

That report will likely land on the National Commission into the Regulation of AI in Healthcare page.