Doctors are still clearly superior when it comes to treatment decisions.

Skin cancer diagnoses using artificial intelligence (AI) are as reliable as those made by medical experts,  but doctors are clearly superior when it came to treatment decisions.

These are the main takeaways from research looking at AI applications under realistic clinical conditions in two skin cancer centres, the University Department of Dermatology at MedUni Vienna and the Sydney Melanoma Diagnostic Centre in Australia. 

The prospective study consisted of two scenarios, with AI being used in scenario A for changes suspicious of skin cancer and in scenario B for patients with many moles. The AI-assisted application was compared in both cases with both medical experts and less experienced physicians.

In scenario A, 172 suspicious pigmented lesions (of which 84 were malignant) were examined in 124 patients; in scenario B, the research team analyzed 5,696 pigmented lesions (of which 18 were malignant) in 66 patients. Two different AI-based smartphone applications were used: a novel 7-class AI algorithm and an ISIC algorithm already used in retrospective preliminary studies. In scenario A, the 7-class AI algorithm showed equivalent diagnostic accuracy compared to the experts while it was significantly superior to the less experienced physicians. The ISIC algorithm, on the other hand, performed significantly worse compared to experts, but better than the inexperienced users.

In terms of treatment decisions, the 7-class algorithm was significantly inferior to the experts but superior to the inexperienced users. The results suggest that an AI-assisted smartphone application for skin cancer diagnosis makes similarly good diagnostic decisions as experts in a real clinical scenario. When it came to treatment decisions, however, the experts were superior to the AI. 

"The AI application tends to remove more benign lesions in the treatment recommendation than experts would,” says study author dermatologist Harald Kittler from MedUni Vienna.  “If you take this into account, the AI application can certainly be used. It should also be borne in mind that if it is used uncritically, too many false-positive findings would have to be clarified." 

The study is published in The Lancet Digital Health.