Learning stochastic object models from imaging measurements using deep generative models

Advanced AmbientGANs for learning stochastic object models (SOMs)

The objective optimization of image-derived statistics, including the test statistic of an observer for specific decision tasks, requires a characterization of all sources of variability in the measured data. To accomplish this, it is necessary to establish a stochastic object model (SOM) that describes the variability within a group of objects to-be imaged. In order for the SOM to be realistic, it is desirable to establish it by use of experimental image data, as opposed to establishing it in a non-data-driven manner. By definition, however, SOMs should be independent of the imaging system, measurement noise, and any reconstruction method employed. Deep learning methods that employ generative adversarial networks (GANs) hold promise for establishing SOMs that describe finitie-dimensional approximations of objects. However, because experimental data recorded by an imaging system represent noisy and indirect measurements of the object, conventional GANs cannot be directly employed for this task. Recently, an augmented GAN architecture named AmbientGAN was proposed that can characterize a distribution of images from noisy and indirect measurements of them and knowledge of the measurement operator. In this work, for the first time, we investigate AmbientGANs for establishing SOMs by use of noisy imaging measurements. Modern multiresolution training approaches, such as employed in the progressive growing of GANs (ProGANs) and style-based GANs (StyGANs), are modified for use in establishing AmbientGANs with high-dimensional medical imaging measurements. The resulting models are referred to as progressive growing AmbientGANs (ProAmGANs) and style-AmbientGANs (StyAmGANs).

An illustration of the proposed modified AmbientGAN architecture. Any advanced GAN architecture employing a progressively growing training procedure can be employed in this framework.

 

The proposed AmbientGAN method can generate clean images when the imaging measurements are contaminated by measurement noise. Both visual examinations and quantitative analyses, including task-specific validations using the Hotelling observer, demonstrated that the proposed AmbientGAN method holds promise to establish realistic SOMs from imaging measurements.

 

Establishing SOMs from fully sampled noisy k-space measurements. (From top row to bottom row) Samples of images generated by ProGAN, ProAmGAN and Sty2AmGAN respectively. The ProGAN was trained with images obtained with the inverse FFT and are demonstrably noisy, while the images generated by the advanced AmbientGAN models (ProAmGAN and Sty2AmGAN) trained directly with k-space data are clean.

Publications

  1. Zhou, W., Bhadra, S., Brooks, F. J., Granstedt, J. L., Li, H., & Anastasio, M. A. (2022, February). Learning stochastic object models from medical imaging measurements by use of advanced ambient generative adversarial networks. J. Med. Imag. 9(1), 015503 (2022).
  2. Zhou, W., Bhadra, S., Brooks, F. J., Granstedt, J. L., Li, H., & Anastasio, M. A. (2021, February). Advancing the AmbientGAN for learning stochastic object models. In Medical Imaging 2021: Image Perception, Observer Performance, and Technology Assessment (Vol. 11599, p. 115990A). International Society for Optics and Photonics.
  3. Zhou, W., Bhadra, S., Brooks, F. J., Li, H., & Anastasio, M. A. (2020, March). Progressively-Growing AmbientGANs for learning stochastic object models from imaging measurements. In Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment (Vol. 11316, p. 113160Q). International Society for Optics and Photonics.
  4. Zhou, W., Bhadra, S., Brooks, F.J., Anastasio, M. A. (2019, February). Learning stochastic object model from noisy imaging measurements using AmbientGANs. In Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment, vol. 10952, pp. 142-148.