The ABCDEs of AI for Skin Cancer Detection

Although discussion of artificial intelligence (AI) in medicine has been ongoing for years, and the technology is making its way into the clinic, there is still much to learn. In fact, given that a hallmark of machine learning is that it can be adaptive, learning will be ongoing for both clinicians and the software as it becomes integrated into clinical practice.

Artificial Intelligence (AI) is a broad term used to describe programs and devices that are designed to “think.” The most popularly cited “definition” of AI is likely that from Stanford University’s John McCarthy¹:

“It is the science and engineering of making intelligent machines, especially intelligent computer programs. It is related to the similar task of using computers to understand human intelligence, but AI does not have to confine itself to methods that are biologically observable.”

Under the broad umbrella of AI is the concept of Machine Learning (ML), which, according to FDA, trains software algorithms to learn from and act on data. Algorithms may be “locked” so that their function does not change or “adaptive,” meaning that behavior can change over time based on new data.

In its examples of artificial intelligence and machine learning technologies, FDA includes, “An imaging system that uses algorithms to give diagnostic information for skin cancer in patients.” (Chan, et al. provide an excellent overview of machine learning that is worth accessing.²)

Convoluted neural networks (CNNs) are a form of deep learning, which is a form of machine learning. CNN was initially designed for image analysis and is thus well-suited for use in dermatology diagnostics. CNN contains two basic operations: convolution and pooling. Zhu et al, explain, “The convolution operation, using multiple filters, is able to extract features (feature map) from the data set, through which their corresponding spatial information can be preserved. The pooling operation, also called subsampling, is used to reduce the dimensionality of feature maps from the convolution operation.”³

“CNN simulates the processing of biological neurons and is the state-of-the-art network for pattern recognition in medical image analysis,” Cockerell et al note, thus offering “advantages over traditional analytical techniques.”⁴

A significant proportion of ML development in health care currently is focused on software as a medical device (SaMD). As opposed to software in a medical device (sometimes called “embedded” software), software as a medical device meets several criteria listed in the sidebar. Most importantly, SaMD is a medical device and includes in vitro diagnostic (IVD) medical devices and can run on general purpose (non-medical purpose) computing platforms.⁵ Two key elements stand out in this description for SaMD: it is fully intended to be diagnostic, and it is distinct from software that may be developed to work exclusively with a specific device, although such software may also incorporate AI.

According to FDA, one of the greatest benefits of AI/ML in software resides in its ability to learn from real-world use and experience, and its capability to improve its performance, known as training and adaptation, respectively.

Currently, FDA has not established a clear regulatory framework specific to SaMD. The agency says it has cleared or approved several AI/ML-based SaMD, but these typically have only included algorithms that are “locked” prior to marketing. In other words, such SaMD are not adaptive.

FDA last fall issued “Artificial Intelligence and Machine Learning (AI/ML) Software as a Medical Device Action Plan” as an initial step in establishing standards for approval of AI-based devices. The publication is not official guidance, but it sets the stage for progress toward a formal framework.

As developments continue in AI in dermatology and especially in skin cancer diagnosis, dermatologists should keep several key considerations in mind.

A. Augmented Intelligence is Ideal

Traditionally, AI has stood for Artificial Intelligence, but across medicine and especially in dermatology, the term Augmented Intelligence is taking hold. Augmented Intelligence reflects the notion that new technologies and evolving hardware and software solutions are not intended to replace clinicians. Rather they are intended to support clinicians by offering additional data and information. In some cases, AI may serve to affirm the dermatologist’s clinical decision making. In other cases, it may point to alternative or additional considerations that may be relevant.

In one of the most favorable findings for AI, a CNN trained by open-source images outperformed the majority (136 of 157) of dermatologists in terms of average specificity and sensitivity. Participating dermatologists had various levels of experience.⁶

But in another analysis of AI use in the detection of potential skin malignancies, researchers concluded, “Good quality AI-based support of clinical decision-making improves diagnostic accuracy over that of either AI or physicians alone, and that the least experienced clinicians gain the most from AI-based support.”⁷

Another analysis of 19 studies identified favorable findings for AI, but the authors suggest caution. Overall, tested AI demonstrated “superior or at least equivalent performance of CNN-based classifiers compared with clinicians.” In all studies, the authors note, AI testing was “in highly artificial settings,” used a single image of the suspicious lesions, and may not have included patient populations and melanoma subtypes encountered in clinical practice.⁸ These findings support the notion that the combined intelligence of the clinician and the software may be optimal.

Tschandl et al describe “human-computer collaboration” in skin cancer diagnosis, finding that, “good quality AI-based support of clinical decision-making improves diagnostic accuracy over that of either AI or physicians alone, and that the least experienced clinicians gain the most from AI-based support. They caution that AI alone could be misleading but that coupled with clinician knowledge, it can improve human diagnosis.⁹

AI performs well in controlled trials, and certain technologies appear poised to improve patient care. As with any technology, much depends on the specific application. CNNs that analyze images do not consider factors such as history that may influence a dermatologist’s assessment of lesions.

B. Be Informed

As technologies emerge, it is incumbent upon dermatologists to understand the specific risks, benefits, and appropriate use of them for skin cancer detection. Tshandl et al caution that poorly designed AI can mislead even experienced diagnosticians.⁹

If “machine learning” is possible, then we must consider how the machine is trained. Ultimately, the utility of any system will rely on the quality of the data used for software training.

There are evolving standards for Good Machine Learning Practice (GMLP, see sidebar). Among important considerations are that: Clinical study participants and data sets are representative of the intended patient population and that training data sets are independent of test sets. These will be important considerations for dermatologists as they vet emerging technologies.

As patients become increasingly savvy about AI, dermatologists must be prepared to explain its proper role in diagnosis to patients. Interestingly, research suggests that even digital native patients will lean on their physicians for guidance on the role and application of AI. In one survey, the majority of individuals under 35 years of age indicated they were ready to accept AI-based diagnostic solutions for early detection of skin cancer, but they indicated a need for increased explainability of AI-based tools.¹⁰

C. Consumer-Facing Hasn’t Caught Up

Just as important—possibly more important—as explanation of AI used in the clinic will be dermatologists’ explanation of consumer-facing mobile apps for skin cancer detection. Several apps are already on the market aimed at helping patients/consumers identify suspicious lesions using AI technology. These are not to be confused with certain telehealth apps that use store-and-forward technology to allow virtual assessment by a dermatologist. AI-based health app developers are not required to seek FDA clearance for their software, and most are likely to attempt to skirt regulation with disclaimers that their products are not intended to diagnose or treat disease.

While the potential for patients to be able to identify suspicious lesions with their phones and then seek early intervention is promising, the technology simply is not yet ready for wide public use. Thinking in terms of “augmented intelligence,” an ideal app will provide additional information to aid diagnosis rather than simply render a diagnosis. For now, patients are best served by their own knowledge and intuition: any new or changing mole requires evaluation; lesions can be assessed based on the ABCDEs. If a patient is suspicious enough to reach for an app, then they ought to see a dermatologist.

One study tested a consumer-facing app’s ability to identify suspicious lesions and benign controls; lesion status was histopathologically confirmed after app assessment. The app had an overall 86.9 percent sensitivity and 70.4 percent specificity. The device used influenced the app’s performance. The sensitivity was significantly higher on the iOS device compared to the Android device (91.0 vs. 83.0 percent).¹¹

A separate study of a different app compared the technology to assessment by a dermatologist. Of 199 lesions, the app was unable to provide an analysis for 90 (45 percent). Among these lesions not analyzed were nine BCC, four atypical nevi, and one lentigo maligna. Among lesions rated by the app as high or medium risk, 67 percent and 79 percent, respectively, were diagnosed as benign nevi or seborrheic keratoses. Interobserver agreement between the app and the dermatologist was poor.¹²

Another study, now six years old—which can be significant in terms of technology—assessed three apps intended to classify lesions as suspicious or benign. Dermatologists pre-screened patients prior to assessment with the app. The apps’ sensitivity and specificity ranged from 21-72 percent and 27-100 percent, respectively, when compared with the specialists’ decisions. Interrater agreement between dermatologists and apps was poor to slight.¹³

D. Diversity Matters

Dermatologists use the ABCDE and seven-point dermoscopic criteria for assessing suspicious lesions. While these criteria are, in a sense, based on recognition of patterns, this is not the same as the pattern recognition inherent in machine learning. The clinician’s approach to assessing lesions and the machine-learning approach are not truly analogous. In fact, while we understand the fundamentals of machine learning, we do not yet fully understand how machine learning in medical imaging takes place; such approaches are deemed “black-box.”

Nonetheless, it is understood that the outcomes from machine learning algorithms depend on the quality of the data from which the algorithm trains. (See sidebar on Good Machine Learning Practice). The majority of clinical images of skin cancer and its mimickers are of individuals with fair skin. Overall, skin of color has been underrepresented in both traditional and online image resources. In an analysis of 15,445 images across six textbooks and two online image resources, only about 20 percent were dark skin individuals; neoplasms of the skin had lowest representation.¹⁴

By all indications, AI developers will be tasked to significantly augment the representation of skin of color in ML training sets. If there is insufficient data from dark skin patients included in the ML process, it will inhibit the ability of the algorithms to identify skin cancers on dark skin.

Given ongoing disparities in skin cancer diagnosis and outcomes among Hispanic and Black patients, it is imperative that future technology advancement seek to close and not widen the care gap.

E. Expansion is Imminent

The question no longer is whether AI will support the diagnosis of melanoma and NMSC in the clinic. The question is when. Existing technologies already show promise. The key to successful integration of AI in the clinic is to optimize outcomes, reduce errors, and limit liability for clinicians who adopt such technology. Those who integrate technologies must fully understand the ML process that supported its development. Users must vet not only the quality of the ML process and its training sets but also the evidence from well-designed trials of the software.

The notion of augmented intelligence encapsulates the ideal approach to use of ML in the clinic. It is not intended to replace assessment by an expert, rather it is intended to support assessment by an expert. In this vein, consumer facing apps intended to detect skin cancers are not currently recommended as they generally have not shown a reliable benefit and cannot replace evaluation by a dermatologist.

1. https://borghese.di.unimi.it/Teaching/AdvancedIntelligentSystems/Old/IntelligentSystems_2008_2009/Old/IntelligentSystems_2005_2006/Documents/Symbolic/04_McCarthy_whatisai.pdf

Accessed March 17, 2022

2. Chan S, Reddy V, Myers B, Thibodeaux Q, Brownstone N, Liao W. Machine Learning in Dermatology: Current Applications, Opportunities, and Limitations. Dermatol Ther (Heidelb). 2020 Jun;10(3):365-386. doi: 10.1007/s13555-020-00372-0. Epub 2020 Apr 6. PMID: 32253623; PMCID: PMC7211783

3. https://www.sciencedirect.com/topics/computer-science/convolutional-neural-networks

Accessed March 18, 2022.

4. Das K, Cockerell CJ, Patil A, Pietkiewicz P, Giulini M, Grabbe S, Goldust M. Machine Learning and Its Application in Skin Cancer. Int J Environ Res Public Health. 2021 Dec 20;18(24):13409.

5. https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-131209-samd-key-definitions-140901.pdf.

6. Brinker TJ, Hekler A, Enk AH, Klode J, Hauschild A, Berking C, Schilling B, Haferkamp S, Schadendorf D, Holland-Letz T, Utikal JS, von Kalle C; Collaborators. Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. Eur J Cancer. 2019 May;113:47-54.

7. Tschandl P, Rinner C, Apalla Z, Argenziano G, Codella N, Halpern A, Janda M, Lallas A, Longo C, Malvehy J, Paoli J, Puig S, Rosendahl C, Soyer HP, Zalaudek I, Kittler H. Human-computer collaboration for skin cancer recognition. Nat Med. 2020 Aug;26(8):1229-1234.

8. Haggenmüller S, Maron RC, Hekler A, Utikal JS, Barata C, Barnhill RL, Beltraminelli H, Berking C, Betz-Stablein B, Blum A, Braun SA, Carr R, Combalia M, Fernandez-Figueras MT, Ferrara G, Fraitag S, French LE, Gellrich FF, Ghoreschi K, Goebeler M, Guitera P, Haenssle HA, Haferkamp S, Heinzerling L, Heppt MV, Hilke FJ, Hobelsberger S, Krahl D, Kutzner H, Lallas A, Liopyris K, Llamas-Velasco M, Malvehy J, Meier F, Müller CSL, Navarini AA, Navarrete-Dechent C, Perasole A, Poch G, Podlipnik S, Requena L, Rotemberg VM, Saggini A, Sangueza OP, Santonja C, Schadendorf D, Schilling B, Schlaak M, Schlager JG, Sergon M, Sondermann W, Soyer HP, Starz H, Stolz W, Vale E, Weyers W, Zink A, Krieghoff-Henning E, Kather JN, von Kalle C, Lipka DB, Fröhling S, Hauschild A, Kittler H, Brinker TJ. Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts. Eur J Cancer. 2021 Oct;156:202-216.

9. Tschandl P, Rinner C, Apalla Z, Argenziano G, Codella N, Halpern A, Janda M, Lallas A, Longo C, Malvehy J, Paoli J, Puig S, Rosendahl C, Soyer HP, Zalaudek I, Kittler H. Human-computer collaboration for skin cancer recognition. Nat Med. 2020 Aug;26(8):1229-1234.

10. Haggenmüller S, Krieghoff-Henning E, Jutzi T, Trapp N, Kiehl L, Utikal JS, Fabian S, Brinker TJ. Digital Natives’ Preferences on Mobile Artificial Intelligence Apps for Skin Cancer Diagnostics: Survey Study. JMIR Mhealth Uhealth. 2021 Aug 27;9(8):e22909.

11. Sangers T, Reeder S, van der Vet S, Jhingoer S, Mooyaart A, Siegel DM, Nijsten T, Wakkee M. Validation of a Market-Approved Artificial Intelligence Mobile Health App for Skin Cancer Screening: A Prospective Multicenter Diagnostic Accuracy Study. Dermatology. 2022 Feb 4:1-8.

12. Chung Y, van der Sande AAJ, de Roos KP, Bekkenk MW, de Haas ERM, Kelleners-Smeets NWJ, Kukutsch NA. Poor agreement between the automated risk assessment of a smartphone application for skin cancer detection and the rating by dermatologists. J Eur Acad Dermatol Venereol. 2020 Feb;34(2):274-278.

13. Ngoo A, Finnane A, McMeniman E, Tan JM, Janda M, Soyer HP. Efficacy of smartphone applications in high-risk pigmented lesions. Australas J Dermatol. 2018 Aug;59(3):e175-e182

14. Alvarado SM, Feng H. Representation of dark skin images of common dermatologic conditions in educational resources: A cross-sectional analysis. J Am Acad Dermatol. 2021 May;84(5):1427-1431