A new computationally efficient method to recover the 3D shape of objects hidden behind a thick scattering layer.
Overview of Confocal Diffuse Tomography
Abstract
Optical imaging techniques, such as light detection and ranging (LiDAR), are essential tools in remote sensing, robotic vision, and autonomous driving. However, the presence of scattering places fundamental limits on our ability to image through fog, rain, dust, or the atmosphere. Conventional approaches for imaging through scattering media operate at microscopic scales or require a priori knowledge of the target location for 3D imaging. We introduce a technique that co-designs single-photon avalanche diodes, ultra-fast pulsed lasers, and a new inverse method to capture 3D shape through scattering media. We demonstrate acquisition of shape and position for objects hidden behind a thick diffuser (≈6 transport mean free paths) at macroscopic scales. Our technique, confocal diffuse tomography, may be of considerable value to the aforementioned applications.
Hardware Prototype
Confocal Scanning. The hardware prototype raster scans the surface of the scattering media, a 25 mm thick piece of opaque foam. A time-resolved detector captures the light that scatters through the media to the hidden object and back to the illuminated point.
Hardware Prototype. The primary components of the hardware prototype consist of a pulsed laser which shares an optical path through a beam splitter with a single-pixel single-photon avalanche diode. The system captures the time of arrival of backscattered photons with a timing resolution on the order of tens of picoseconds.
D.B. Lindell, G. Wetzstein. Three Dimensional Imaging Based on Confocal Diffuse Tomography. Nature Communications, 2020
BibTeX
@article{Lindell:2020:CDT,
title={Three-dimensional imaging through scattering media based on confocal diffuse tomography},
author={Lindell, David B and Wetzstein, Gordon},
journal={Nat. Commun.},
volume={11},
number={4517},
year={2020},
publisher={Nature Publishing Group}
}
Imaging through scattering media
Schematic of 3D Imaging through scattering media. a A pulsed laser and timeresolved single-photon detector raster-scan the surface of the scattering medium. b Light diffuses through the medium, is back-reflected by the hidden object, and diffuses back through the medium to the detector. c Returning photons from the hidden object are captured by the detector over time, with earlier arriving photons being gated out (dashed line). SG, scanning galvanometer; BS, beam splitter; OL, objective lens; SPAD, single-photon avalanche diode; TCSPC, time-correlated single-photon counter.
Overview of reconstruction procedure. The captured time-resolved measurements (a) are deconvolved with a calibrated diffusion operator to estimate a volume without the effects of diffusive scattering (b). Applying a confocal inverse filter recovers the hidden retroreflective letter ‘S’ (c), which resembles a photograph of the hidden scene (c, bottom).
Experimental Results
Scene Photo
Measurements
Gated Imaging
Confocal Diffuse Tomography
Experimental results (Letters “UT”). The hidden scene is composed of two retroreflective letters (top left). The shape of the letters is completely obscured in the time-resolved measurements, which directly visualize the scattered light returning through the media (top right). Even when time-gating the measurements with a 50 ps window, the shape is obscured (bottom left). Using confocal diffuse tomography, the shape and position of the letters can be recovered (bottom right).
Scene Photo
Measurements
Gated Imaging
Confocal Diffuse Tomography
Experimental results (Traffic Cones). The hidden scene is composed of three traffic cones (top left). Again, the scene is completely obscured in the time-resolved measurements (top right), even when gating the measurements in a 50 ps time slice (bottom left). Confocal diffuse tomography recovers the positions of the cones (bottom right).
Scene Photo
Measurements
Gated Imaging
Confocal Diffuse Tomography
Experimental results (Mannequin). Here, the hidden scenes is a reflective mannequin (top left). Scattering completely obscures the scene in the time-resolved measurements (top right), even when gating the measurements with a 50 ps window (bottom left). Confocal diffuse tomography recovers the shape of the mannequin (bottom right).
Scene Photo
Measurements
Gated Imaging
Confocal Diffuse Tomography
Experimental results (Diffuse Letter). In this case, the scene is a diffuse hidden letter (top left). While the shape of the scene is not visible in the time-resolved measurements because of scattering (top right, bottom left), confocal diffuse tomography recovers its shape (bottom right).
Acknowledgements
This project is supported by a Stanford Graduate Fellowship in Science and Engineering, a National Science Foundation CAREER award (IIS 1553333), a Sloan Fellowship, the DARPA REVEAL program, a PECASE form the ARO, and by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant.
Related Projects
You may also be interested in related projects, where we have developed non-line-of-sight imaging systems:
