GasHis-Transformer: A Multi-scale Visual Transformer Approach for Gastric Histopathological Image Detection

Abstract: In this paper, a multi-scale visual transformer model, referred as GasHis-Transformer, is proposed for Gastric Histopathological Image Detection (GHID), which enables the automatic global detection of gastric cancer images. GasHis-Transformer model consists of two key modules designed to extract global and local information using a position-encoded transformer model and a convolutional neural network with local convolution, respectively. A publicly available hematoxylin and eosin (H&E) stained gastric histopathological image dataset is used in the experiment. Furthermore, a Dropconnect based lightweight network is proposed to reduce the model size and training time of GasHis-Transformer for clinical applications with improved confidence. Moreover, a series of contrast and extended experiments verify the robustness, extensibility and stability of GasHis-Transformer. In conclusion, GasHis-Transformer demonstrates high global detection performance and shows its significant potential in GHID task.

• The paper proposes GasHis-Transformer, a novel hybrid CNN-Visual Transformer model for Gastric Histopathological Image Detection (GHID) that captures both local and global features.
• GasHis-Transformer integrates a Global Information Module based on BoTNet-50 and a Local Information Module inspired by Inception-V3 to effectively analyze histopathological images.
• Experimental results show GasHis-Transformer achieved 97.97% accuracy on GHID, demonstrated robustness against noise and attacks, and generalized well to other multi-class cancer datasets.

Overview of "GasHis-Transformer: A Multi-scale Visual Transformer Approach for Gastric Histopathological Image Detection"

The paper proposes GasHis-Transformer, an advanced model forged by integrating Convolutional Neural Networks (CNNs) and Visual Transformers (VTs), tailored specifically for Gastric Histopathological Image Detection (GHID). This multi-scale visual transformer addresses the shortcomings inherent in CNNs, namely their inadequacy in handling global information, by exploiting the capabilities of transformers. The core hypothesis is that the hybrid model can leverage CNN's efficiency in local feature extraction while simultaneously utilizing the VTs' strength in capturing broader, archival features of gastric histopathological images.

Key Components and Methodology

The GasHis-Transformer is strategically composed of two pivotal modules:

1. Global Information Module (GIM): Modeled after BoTNet-50, this module replaces the final convolutional layers of the ResNet-50 network with multiple-head self-attention (MHSA) layers. The MHSA layers are adept at capturing the global context owing to their use of relative position encoding, which facilitates understanding of global features within histopathological images.
2. Local Information Module (LIM): This module draws upon Inception-V3's ability to analyze multi-scale information. It captures critical local information indicative of nuanced changes and details in histopathological images — features that are imperative for the precise classification of regions with minute pathological changes.

The integrated model performs end-to-end learning and utilizes image normalization to enhance the speed and accuracy of convergence by scaling pixel intensity distributions. GasHis-Transformer not only fuses the capabilities of GIM and LIM but also employs a robust optimization layer that utilizes Dropout for GasHis-Transformer and Dropconnect for its lightweight variant, LW-GasHis-Transformer.

Experimental Evaluation and Results

The paper demonstrates the model's efficacy using several key experiments, each aimed at revealing the robustness and utility of GasHis-Transformer:

• GHID Performance: Engineered on a dataset of Hematoxylin and Eosin (H&E) stained images, GasHis-Transformer achieved high accuracy (97.97%) in distinguishing between normal and abnormal gastric tissues, outperforming prominent existing models like Xception and ResNet-50.
• Robustness Testing: Tests involving adversarial attacks (e.g., FGM, FSGM) and conventional noises (Gaussian, Salt & Pepper) highlighted the model's resilience, particularly against FGM and uniform noise, underscoring its reliability in practical scenarios.
• Generalization in Multi-Class Cancer Detection: When extended to other datasets including BreakHis and IHC-LI-DS, the model maintained high classification accuracy, demonstrating its applicability beyond gastric cancer, to other histopathological contexts such as breast cancer and lymphoma.

Implications and Future Directions

The results indicate that combining CNNs and VTs into a unified framework yields significant improvements in GHID, capitalizing on the unique strengths of each approach. This advancement paves the way for enhanced accuracy in medical image analysis, which is crucial for precise diagnostics and treatment planning.

Given the complexities associated with histopathological images including high variance in detail and similarity between categories, GasHis-Transformer's design presents a promising advancement in medical image classification tasks. Moreover, the lightweight version, LW-GasHis-Transformer, offers an efficient alternative for clinical applications with resource constraints.

Future research is poised to explore the integration of domain adaptation techniques and few-shot learning to further enhance model performance on limited and diverse datasets, ensuring adaptability and precision across various medical imaging modalities. This trajectory promises enhanced support tools for medical practitioners, lending to faster, more accurate diagnostic workflows.