Training generative adversarial networks for optical property mapping using synthetic image data

Osman, A.; Crowley, J.; Gordon, G. S. D

Abstract

We demonstrate the training of a generative adversarial network (GAN) for the prediction of optical property maps (scattering and absorption) using spatial frequency domain imaging (SFDI) image data sets that are generated synthetically with a free open-source 3D modelling and rendering software, Blender. The flexibility of Blender is exploited to simulate 5 models with real-life relevance to clinical SFDI of diseased tissue: flat samples containing a single material, flat samples containing 2 materials, flat samples containing 3 materials, flat samples with spheroidal tumours and cylindrical samples with spheroidal tumours. The last case is particularly relevant as it represents wide-field imaging inside a tubular organ e.g. the gastro-intestinal tract. In all 5 scenarios we show the GAN provides an accurate reconstruction of the optical properties from single SFDI images with a mean normalised error ranging from 1.0-1.2% for absorption and 1.1%-1.2% for scattering, resulting in visually improved contrast for tumour spheroid structures. This compares favourably with the ∼10% absorption error and ∼10% scattering error achieved using GANs on experimental SFDI data. Next, we perform a bi-directional cross-validation of our synthetically-trained GAN, retrained with 90% synthetic and 10% experimental data to encourage domain transfer, with a GAN trained fully on experimental data and observe visually accurate results with an error of 6.3%-10.3% for absorption and 6.6%-11.9% for scattering. Our synthetically trained GAN is therefore highly relevant to real experimental samples but provides the significant added benefits of large training datasets, perfect ground-truths and the ability to test realistic imaging geometries, e.g. inside cylinders, for which no conventional single-shot demodulation algorithms exist. In the future, we expect that the application of techniques such as domain adaptation or training on hybrid real-synthetic datasets will create a powerful tool for fast, accurate production of optical property maps for real clinical imaging systems.

• Journal Article Type: Article
• Acceptance Date: Jun 13, 2022
• Online Publication Date: Sep 8, 2022
• Publication Date: Sep 8, 2022
• Deposit Date: Oct 19, 2023
• Publicly Available Date: Oct 19, 2023
• Journal: Biomedical Optics Express
• Electronic ISSN: 2156-7085
• Publisher: Optica Publishing Group
• Peer Reviewed: Peer Reviewed
• Volume: 13
• Issue: 10
• Pages: 5171-5186
• DOI: https://doi.org/10.1364/boe.458554
• Keywords: Atomic and Molecular Physics, and Optics; Biotechnology
• Public URL: https://nottingham-repository.worktribe.com/output/10920199
• Publisher URL: https://opg.optica.org/boe/fulltext.cfm?uri=boe-13-10-5171&id=500222
• Additional Information: This article is maintained by: Optica Publishing Group; Crossref DOI link to publisher maintained version: https://doi.org/10.1364/BOE.458554; Article type: research-article; Similarity check: Screened by Similarity Check; Peer reviewed: Yes; Review process: Single blind; Received: 17 March 2022; Accepted: 13 June 2022; Published: 8 September 2022; Copyright: Published by Optica Publishing Group under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.