Hybrid GAN-DIFFUSION Framework for Ultra sound to CT Translation: A Residual Refinement Approach

Abstract

The translation of 3D ultrasound (US) to computed tomography (CT) is a significant challenge in medical imaging due to the vast domain gap and inherent US artifacts. This thesis introduces a novel, two-stage hybrid framework to address this, synergizing a custom Generative Adversarial Network (GAN) with a Denoising Diffusion Probabilistic Model (DDPM). The first stage confronts GAN instability and mode collapse with our proposed UNetSelfCritique3D generator, which uses an internal feature-matching loss to enforce structural consistency and produce a coarse but anatomically faithful CT volume. The second stage employs a conditional DDPM for residual refinement, using the coarse GAN output as a strong prior to synthesize missing high-frequency details. Validated on the clinical TRUSTED dataset, our baseline 3D Pix2Pix model achieved a Peak Signal-to-Noise Ratio (PSNR) of 11.34 dB and a Structural Similarity Index (SSIM) of 0.181. The UNetSelfCritique3D significantly improved performance to 20.57 dB PSNR and 0.468 SSIM, demonstrating effective mitigation of mode collapse. The final hybrid GAN-Diffusion model achieved the best results with 21.10 dB PSNR, 0.456 SSIM, and 0.547 Normalized Cross-Correlation (NCC), representing a substantial 86% improvement in PSNR over the baseline. This work establishes the first crucial performance benchmark for 3D US-to-CT translation and validates the feasibility of our hybrid approach. The proposed framework and code are made publicly available to foster reproducibility and accelerate future research.