Roll your own loss function

Reading: Inside the maths that drives AI, Nature technology feature, 3 July 2024.

Loss functions are how you measure error, or progress, in machine learning. The game is to minimize the loss (error) from predictions against ground truth data, without going too far and memorizing all the examples.

There are standard ways to do this, but you can also roll your own. I’ve not really considered this before. I have when using GAs to evolve solutions, but I’ve not plugged in a totally custom loss function into, say, a tree learner.

I do have a use case for this. In a medical diagnostic support context, you really, strongly, want to capture every positive case (e.g., cancer) and are happy to trade a few false positives for this.

What I mean is: say if you have a physical diagnostic test, you could just examine everyone with symptoms. You will catch all positive cases, but that’s not great for the patient, and it costs money.

If you have a software diagnostic that can reliably triage positive cases for the physical exam, that going to save a bunch of resources. But you’ll want to lean heavily on the side of caution to avoid missing cases.

It’s the precision-recall tradeoff, where I want strong recall. Very strong. Could be a case for a custom loss function.

And from a useful review article , it may be something derived from F2 is what I’m looking for:

Like the F1 score, the F2 score ranges from 0 to 1, with a higher score indicating better performance. However, the F2 score places a greater emphasis on recall, making it useful when it is important to minimize false negatives.

Maybe I should roll my own loss function using those ideas.