深度学习重建红细胞光学断层3D结构

采用可见光源代替X射线进行光学衍射断层成像(ODT),可以无创地测定样品定量折射率(RI)的三维分布。ODT系统中采用倾斜照明的断层扫描架构对样品进行采样,并用高数值孔径(NA)物镜收集尽可能多的散射光子,可实现亚波长空间分辨率的RI成像,在细胞生物学研究中有广泛的用途。ODT照明采样原理如图1a-b所示,可以是光束旋转扫描(图1a)或样品旋转扫描(图1b)。实际实验中,入射光束或样品的旋转断层光学取样总是受限于物镜有限的NA或成像光学系统的工作距离。因此,在特定旋转方向上的散射光空间频率(图1c-d)在频谱域中是受限的。

ODT通过将不同旋转方向的散射光频谱信息拼接在一起,重建RI的定量三维分布。而如图1e-f所示,在空间频率域中沿光照或者样品旋转轴产生一系列丢失数据,即所谓的“漏锥”问题。因此,导致重建的三维图像变形和模糊,使得RI图像分割和定量分析变得困难。
 

图1. 光学衍射断层成像(ODT)“漏锥”示意图。(a)带旋转照明和(b)旋转样品的ODT示意图。(c-d)目标捕获的空间散射光频谱: 红色箭头表示投射在球冠上的透射和散射信号,目标的数值孔径限制了散射信号的带宽。(e-f) ODT 将不同旋转方向的空间光频谱帽进行合成,重建样品的三维空间光频谱。在照明旋转(e)或样品旋转(f)都会遇到沿旋转轴的缺失光频谱问题。

解决“漏锥”问题是拓宽ODT 在细胞生物学研究和其他亚波长结构三维成像中应用的关键步骤。针对“漏锥”问题,利用细胞结构的稀疏性,已有研究人员提出了几种基于模型的迭代重建方法[1,2]。为了与测量一致,这些模型需包含先验信息,例如RI上的非负性约束。然而,在真实、复杂的细胞样本中,没有约束条件是具备普适性的。因此,目前的挑战仍然是根据特定样品或者细胞的成像特性找到合适的约束条件。

在Advanced Photonics 2020年第2期的文章中,借鉴深层神经网络(DNN)训练,瑞士洛桑联邦理工学院的研究人员提出了一种重建红细胞三维结构的DNN,解决了ODT中的“漏锥”问题(Joowon Lim, Ahmed B. Ayoub, Demetri Psaltis. Three-dimensional tomography of red blood cells using deep learning[J]. Advanced Photonics, 2020, 2(2): 026001)。该DNN训练基于数字构造的虚拟模型,创造性地为训练 DNN 提供了准确真实的信息。该网络以人工重建的数字化模型为初始模型,从数字“真实信息”中提取传统基于Rytov近似重建的缺失特征,使新模型比以前的模型更加精确。因此,由这些合成模型训练的网络能够从实验测量的数据中成功地重建出小鼠红细胞的图像,从而大大提高了图像质量和分辨率。为了进一步验证结果,论文中将重建结果产生的半合成测量值与实验数据进行了比较, 得到了非常吻合的一致性。

作者提出利用 DNN 找到特定细胞类型特定约束的创新想法,为解决“漏椎”问题提供了新途径。高保真度的红细胞图像重建有助于对血细胞的定量分析,尤其是在流式细胞仪应用中。基于DNN的图像重建还可以进一步适用于结合ODT的多模态成像系统,其中的关联成像数据可以用于支撑DNN训练的可验性。

参考文献

1. Lim, J., et al., Comparative study of iterative reconstruction algorithms for missing cone problems in optical diffraction tomography. Optics Express, 2015. 23(13): p. 16933-16948.

2. Sung, Y. and R.R. Dasari, Deterministic regularization of three-dimensional optical diffraction tomography. Journal of the Optical Society of America A, 2011. 28(8): p. 1554-1561.

 

Solving the missing cone problem by deep learning

Abstract

A commentary on the article “Three-dimensional tomography of red blood cells using deep learning” by J. Lim, A. Ayoub, and D. Psaltis, Adv. Photonics Volume 2, Issue 2, doi: 10.1117/1.AP.2.2.026001.

 

In order to extract the quantitative three-dimensional (3-D) distribution of refractive index (RI) in live cells noninvasively, optical diffraction tomography (ODT) uses the non-ionizing light sources instead of x-rays to perform a computational holographic tomography. To resolve the cellular structures with sub-wavelength resolution, the cells are sampled using tomographic scanning with oblique illumination and as many as possible scattered photons are collected with high numerical aperture (NA) objective as shown in Figs. 1(a) and 1(b). Practically, the tomographic rotation of incident beam [Fig. 1(a)] or sample [Fig. 1(b)] is always confined by a limited NA of the objective or a working distance of the imaging optics. The scattering spectrum [Figs. 1(c) and 1(d)] at certain rotation direction is, therefore, bounded at high spatial frequencies. ODT reconstructs the 3-D distribution of RI by casting scattering spectrum caps from different rotation directions together, which leads to a range of missing data in a spatial frequency domain along the rotation axis of illumination or a sample: the so-called "missing cone." For this reason, the reconstructed 3-D images are distorted and dim, making the segmentation and quantitative analysis difficult. Solving the missing cone problem is a crucial step for widening ODT applications in cell biology research and 3-D imaging of other subwavelength structures.

 

Schematic diagram of "missing cone" in optical diffractive tomography (ODT). (a), (b) Schematic diagram of ODT with rotating illumination (a) and rotating sample (b). (c), (d) The spatial scattering spectrum captured by the objective: red arrows indicate the transmitted and scattered signal projected on a spherical cap; the numerical aperture of the objective limits the angular bandwidth of the scattered signal. (e), (f) ODT casts spatial spectrum caps from different rotation directions together in order to reconstruct the 3-D spatial spectrum of the sample. The missing spectrum along the rotation axis is observed with both illumination rotation (e) and sample rotation (f).

To address the missing cone problem, several model-based iterative reconstruction methods were proposed by taking advantage of the sparse nature of cellular structures.1,2 In order to be consistent with measurements, these models incorporate prior information, such as, for example, non-negativity constraints on RI. However, no individual constraints can always be satisfied in real, complex cellular samples. The challenge remains to find an appropriate constraint according to the characteristics of specific cells.

In this issue of Advanced Photonics, Lim et al.3 present a deep neural network (DNN) to reconstruct the 3-D structure of red blood cells (RBCs). The progress is made in solving the missing cone problem in ODT by learning from DNN training based on the digitally constructed phantom models, which creatively provide the accurate ground truth for training the DNN. By using a digital artificial reconstruction as an initial model, the network extracts the missing features of the traditional Rytov reconstruction from the digital "ground truth" which makes a new model more accurate than previous. As a result, the network trained by these synthetic phantoms is able to successfully reconstruct an image of a mouse RBC from the experimentally measured data, resulting in image quality and resolution greatly improved over the previous methods. To further validate the result, the semisynthetic measurements generated from the reconstructed result were compared with the experimental data. Excellent consistency was obtained.

The innovative idea of finding a specific constraint with respect to specific cell types by using the DNN as proposed by Lim et al. promotes an intriguing possibility for tackling the challenge of the missing cone problem that has plagued OTD since its inception. The image reconstruction of RBCs with high fidelity can be particularly useful for quantitative analysis of blood cells, especially in flow cytometer applications.4 DNN-based image reconstructions can also be further adapted in multi-modal imaging systems equipped with ODT,5 where correlative imaging data can be used to further support the experimental ground truth for DNN training.

References

1.Lim, J., et al., Comparative study of iterative reconstruction algorithms for missing cone problems in optical diffraction tomography. Optics Express, 2015. 23(13): p. 16933-16948. https://doi.org/10.1364/OE.23.016933

2.Sung, Y. and R.R. Dasari, Deterministic regularization of three-dimensional optical diffraction tomography. Journal of the Optical Society of America A, 2011. 28(8): p. 1554-1561. https://doi.org/10.1364/JOSAA.28.001554

3. J. Lim, A. Ayoub and D. Psaltis, “Three-dimensional tomography of red blood cells using deep learning,” Adv. Photonics, 2 (2), 026001 (2020). https://doi.org/10.1117/1.AP.2.2.026001

4. F. Merola et al., “Tomographic flow cytometry by digital holography,” Light Sci. Appl., 6 e16241 (2017). https://doi.org/10.1038/lsa.2016.241

5. D. Dong et al., “Super-resolution fluorescence-assisted diffraction computational tomography reveals the three-dimensional landscape of the cellular organelle interactome,” Light Sci. Appl., 9 11 (2020). https://doi.org/10.1038/s41377-020-0249-4

Biography

Dashan Dong received his BS degree in applied physics from Nankai Universiy, Tianjin, China, in 2014 and his PhD in optics, from Peking University, Beijing, China, in 2019. He is now a postdoc at Institute of Modern Optics, School of Physics, Peking University. His current research focuses on the application of optical diffractive tomography in bio-medical research.

Kebin Shi works in the field of optical imaging and spectroscopy for biophotonics and precision metrology. His recent research interests include super-resolution imaging, nonlinear holography and femtosecond frequency comb metrology. His research publication includes over 80 technical papers and 6 patents.