Schlieren Photography with Light Field Probes | ICCP 2011, IJCV 2014

Gordon Wetzstein, Ramesh Raskar, Wolfgang Heidrich

A new approach to capturing refraction in transparent media: Light Field Background Oriented Schlieren Photography.

$Light field probes -when included into the background of a scene- allow otherwise invisible optical properties to be photographed. In this example, the probe contains a classic Rainbow Schlieren filter that codes the angles and magnitudes of complex refractive events in hue and saturation.$

Light field probes -when included into the background of a scene- allow otherwise invisible optical properties to be photographed. In this example, the probe contains a classic Rainbow Schlieren filter that codes the angles and magnitudes of complex refractive events in hue and saturation.

Abstract

We introduce a new approach to capturing refraction in transparent media, which we call Light Field Background Oriented Schlieren Photography (LFBOS). By optically coding the locations and directions of light rays emerging from a light field probe, we can observe and measure changes of the refractive index field between the probe and a camera or an observer. Rather than using complicated and expensive optical setups as in traditional Schlieren photography we employ commodity hardware; our prototype consists of a camera and a lenslet array. By carefully encoding the color and intensity variations of a 4D probe instead of a diffuse 2D background, we avoid expensive computational processing of the captured data, which is necessary for Background Oriented Schlieren imaging (BOS).We analyze the benefits and limitations of our approach and discuss application scenarios.

Best Paper Award

This paper won the Best Paper Award at the International Conference on Computational Photography (ICCP) in 2011.

FIles

ICCP 2011 paper (pdf) (bib)
ICCP 2011 supplemental material (pdf)
IJCV 2014 journal paper (link)
Video (mov)
Presentation (ppt)

Results

Conceptual sketch of hand-held Schlieren photography.

$A lenticular light field probe encoding 1D directional light variation of the refraction created by a lens with an intensity gradient. This type of filter resembles the knife edge filter of traditional Schlieren photography in the specific direction. Left: probe encoding a horizontal intensity gradient without any object; center left: convex lens with uniform background illumination; center right: lens in front of the probe; right: lens in front of the rotated probe.$

A lenticular light field probe encoding 1D directional light variation of the refraction created by a lens with an intensity gradient. This type of filter resembles the knife edge filter of traditional Schlieren photography in the specific direction. Left: probe encoding a horizontal intensity gradient without any object; center left: convex lens with uniform background illumination; center right: lens in front of the probe; right: lens in front of the rotated probe.

$A plate in front of a uniform background (left), and a light field probe that encodes directional variation caused by refraction with a horizontal (center left) and a vertical (center right) intensity gradient. The magnifications (right) and the structure on the plate show how otherwise invisible information is revealed with our probes. The color bar, mapping colors to magnitudes of refraction, is computed from the field of view of the lenticulars and a calibration gradient that is cropped from the photographs.$

A plate in front of a uniform background (left), and a light field probe that encodes directional variation caused by refraction with a horizontal (center left) and a vertical (center right) intensity gradient. The magnifications (right) and the structure on the plate show how otherwise invisible information is revealed with our probes. The color bar, mapping colors to magnitudes of refraction, is computed from the field of view of the lenticulars and a calibration gradient that is cropped from the photographs.

A variety of directional filters can be encoded in our probes. From left: uniform background, annular bright field, annular dark field, circular intensity gradient, horizontal cutoff, and vertical cutoff.

$The classic Rainbow Schlieren filter encoded in our light field probe can visualize magnitudes and angles of refractions in complex media such as this mix of clear corn syrup and water.$

The classic Rainbow Schlieren filter encoded in our light field probe can visualize magnitudes and angles of refractions in complex media such as this mix of clear corn syrup and water.

$Light Field Background Oriented Schlieren photography compared to a failure case of background oriented Schlieren imaging. Optical flow algorithms in BOS require the background to be focused, which places the object out-of-focus (upper left). In this case, the refractions are so strong that they blur out the background pattern and therefore prevent a reliable optical flow estimation (upper right). Our LFBOS works for in-focus (lower left) and out-of-focus settings (lower right).$

Light Field Background Oriented Schlieren photography compared to a failure case of background oriented Schlieren imaging. Optical flow algorithms in BOS require the background to be focused, which places the object out-of-focus (upper left). In this case, the refractions are so strong that they blur out the background pattern and therefore prevent a reliable optical flow estimation (upper right). Our LFBOS works for in-focus (lower left) and out-of-focus settings (lower right).