Skin Lesion Classification Using Ensembles of Multi-Resolution EfficientNets with Meta Data

Abstract: In this paper, we describe our method for the ISIC 2019 Skin Lesion Classification Challenge. The challenge comes with two tasks. For task 1, skin lesions have to be classified based on dermoscopic images. For task 2, dermoscopic images and additional patient meta data have to be used. A diverse dataset of 25000 images was provided for training, containing images from eight classes. The final test set contains an additional, unknown class. We address this challenging problem with a simple, data driven approach by including external data with skin lesions types that are not present in the training set. Furthermore, multi-class skin lesion classification comes with the problem of severe class imbalance. We try to overcome this problem by using loss balancing. Also, the dataset contains images with very different resolutions. We take care of this property by considering different model input resolutions and different cropping strategies. To incorporate meta data such as age, anatomical site, and sex, we use an additional dense neural network and fuse its features with the CNN. We aggregate all our models with an ensembling strategy where we search for the optimal subset of models. Our best ensemble achieves a balanced accuracy of 74.2% using five-fold cross-validation. On the official test set our method is ranked first for both tasks with a balanced accuracy of 63.6% for task 1 and 63.4% for task 2.

• The paper introduces an ensemble of multi-resolution EfficientNets that significantly enhances skin lesion classification performance.
• It integrates patient meta data through a dual-path architecture, effectively fusing image and clinical information.
• The approach achieves a cross-validation balanced accuracy of 74.2%, underscoring its potential for advanced diagnostic applications.

Skin Lesion Classification Using Ensembles of Multi-Resolution EfficientNets with Meta Data

The paper "Skin Lesion Classification Using Ensembles of Multi-Resolution EfficientNets with Meta Data" explores a sophisticated approach to the classification of skin lesions, developed in the context of the ISIC 2019 Skin Lesion Classification Challenge. The challenge comprises two primary tasks: classification based solely on dermoscopic images, and classification augmented with additional patient meta data including age, anatomical site, and sex.

Methodological Approach

The authors introduce a method that leverages the power of multi-resolution EfficientNets, a family of CNNs known for their superior performance across various computer vision tasks. The EfficientNets are employed in combination with extensive image preprocessing, including cropping and color constancy adjustments, to handle variability in image quality and resolution. An intricate ensemble strategy, optimizing the configuration and subset of models, is implemented to bolster classification performance.

A key aspect of the method is the incorporation of meta data through a dual-path architecture for the second task. Here, a dense neural network processes the meta data and its features are subsequently fused with those output by the CNN branch. The approach addresses common challenges such as class imbalance through loss balancing, and deploys data augmentation techniques extensively to ensure robustness, while a data-driven strategy injects additional benign alterations classes to tackle the unknown classes challenge.

Numerical Results

The ensemble method achieves a cross-validation balanced accuracy of 74.2%, with testing accuracies recorded at 63.6% for task 1 and 63.4% for task 2. Despite a notable drop in test performance compared to cross-validation, the model's robustness and accuracy underscore its effectiveness and state-of-the-art caliber within the competition framework.

Observations and Implications

The research underscores the significance of multi-resolution approaches and diverse model architecture ensembles in achieving optimal performance in medical image classification tasks. Notably, EfficientNets exhibit adaptability to medical imaging challenges, particularly when integrated with comprehensive pre-processing and augmentation strategies. However, the task of accurately identifying an unknown new class remains challenging, with test results reflecting lower sensitivities and specificities in those instances.

As for meta data integration, while improvements in cross-validation performance were documented, the challenge of effectively harnessing incomplete or sparse data sets still poses difficulties, especially when predicting new or underrepresented classes. Addressing these issues will be crucial for future research aimed at enhancing the utility of AI in dermatological diagnostics.

Future Directions

The outcomes presented in this study open pathways for further exploration into using EfficientNets across varied medical imaging tasks. Future work could explore the generalization of the architectures across different datasets and develop more robust strategies for integrating meta data, possibly utilizing techniques such as meta-learning. Furthermore, the development of more nuanced methods to handle unknown or rare classes, potentially through semi-supervised learning or anomaly detection frameworks, may present new avenues to refine model performance in real-world settings.

Overall, this research highlights the technical sophistication necessary to advance automated diagnostic systems in dermatology and emphasizes the continuing need for innovation in handling the intricate challenges posed by real-world medical data.