Image Synthesis with Disentangled Attributes for Chest X-Ray Nodule Augmentation and Detection

Abstract: Lung nodule detection in chest X-ray (CXR) images is common to early screening of lung cancers. Deep-learning-based Computer-Assisted Diagnosis (CAD) systems can support radiologists for nodule screening in CXR. However, it requires large-scale and diverse medical data with high-quality annotations to train such robust and accurate CADs. To alleviate the limited availability of such datasets, lung nodule synthesis methods are proposed for the sake of data augmentation. Nevertheless, previous methods lack the ability to generate nodules that are realistic with the size attribute desired by the detector. To address this issue, we introduce a novel lung nodule synthesis framework in this paper, which decomposes nodule attributes into three main aspects including shape, size, and texture, respectively. A GAN-based Shape Generator firstly models nodule shapes by generating diverse shape masks. The following Size Modulation then enables quantitative control on the diameters of the generated nodule shapes in pixel-level granularity. A coarse-to-fine gated convolutional Texture Generator finally synthesizes visually plausible nodule textures conditioned on the modulated shape masks. Moreover, we propose to synthesize nodule CXR images by controlling the disentangled nodule attributes for data augmentation, in order to better compensate for the nodules that are easily missed in the detection task. Our experiments demonstrate the enhanced image quality, diversity, and controllability of the proposed lung nodule synthesis framework. We also validate the effectiveness of our data augmentation on greatly improving nodule detection performance.

• The paper presents a novel synthesis method that uses GANs and gated convolutions to generate realistic lung nodule shapes, sizes, and textures.
• It incorporates precise size modulation and a hard example mining strategy to augment training data for improved CAD detection accuracy.
• Experimental results using metrics like PSNR, SSIM, and FID demonstrate significant performance improvements over existing state-of-the-art methods.

Image Synthesis with Disentangled Attributes for Chest X-Ray Nodule Augmentation and Detection

Introduction

The paper introduces a novel lung nodule synthesis framework to aid Computer-Assisted Diagnosis (CAD) systems in effectively detecting lung nodules from chest X-ray (CXR) images. Given the constrained availability of high-quality annotated datasets, which are critical for robust CAD system training, the synthesis framework aims to boost data augmentation by creating realistic lung nodules with controllable attributes such as shape, size, and texture. The proposed method employs Generative Adversarial Networks (GANs) for shape generation and leverages gated convolutions for texture synthesis.

Lung Nodule Synthesis Framework

The framework consists of three primary components: Shape Generation, Size Modulation, and Texture Synthesis.

Shape Generation

The Shape Generator uses a DCGAN architecture to produce diverse shape masks, crucial for modeling realistic lung nodule contours. It starts with sampling a latent vector from a Gaussian distribution and employs transposed convolutions to map this vector into a 128×128 binary mask.

Figure 1: The detailed structure of the proposed Shape Generator in the Shape Generation step.

Size Modulation

Size modulation is introduced to alter the nodule size precisely at pixel-level granularity. This adjustment uses morphological operations based on image scaling to modulate the diameter of a nodule shape, ensuring quantitative control over nodule dimensions.

Texture Synthesis

The Texture Generator synthesizes visually plausible nodule textures conditioned by the modulated shape masks. Leveraging gated convolutional layers, the generator utilizes the contextual information of surrounding anatomical structures to render realistic nodules. The generator's architecture follows a coarse-to-fine method where a coarse generator first synthesizes generalized textures, refined subsequently by a refinement generator to enhance image fidelity.

Figure 2: The architecture of the proposed Texture Generator in the Texture Synthesis step.

Hard Example Mining (HEM) Strategy

To further enhance nodule detection performance, the paper proposes an HEM-based augmentation strategy. This method synthesizes lung nodules by focusing on attributes that are often missed by existing detectors, thereby training CAD systems with targeted data that improve sensitivity and accuracy.

Figure 3: The proposed HEM-based data augmentation strategy consists of four steps to enhance detection performance.

Experimental Results

The evaluation showcases the framework's superiority in generating high-quality and diverse synthetic lung nodules. Quantitative metrics such as PSNR, SSIM, and FID demonstrate substantial improvements over existing methods like EdgeConnect and StructureFlow. Qualitative comparisons further reinforce the visually realistic outputs achieved by the proposed synthesis method.

Figure 4: Qualitative comparison between state-of-the-art methods and the proposed method for different nodule patch cases.

Conclusion

The proposed lung nodule synthesis framework offers significant advancements in image quality and controlling nodule attributes for effective data augmentation. It also introduces a unique HEM strategy that efficiently improves lung nodule detection capabilities in CAD systems. Future work may explore extending the application of the synthesis framework to other fields within medical imaging and further refine the control over nodule attributes such as texture and appearance. Potential applications in virtual reality training and counterfactual explanation for CAD systems present promising directions for ongoing research.