Holographic Parallax Improves 3D Perceptual Realism | SIGGRAPH 2024

Dongyeon Kim*, Seung-Woo Nam*, Suyeon Choi*, Jong-Mo Seo, Gordon Wetzstein, Yoonchan Jeong

Inclusion of parallax cues in CGH rendering plays a crucial role in enhancing perceptual realism.

ABSTRACT

Holographic near-eye displays are a promising technology to solve long-standing challenges in virtual and augmented reality display systems. Over the last few years, many different computer-generated holography (CGH) algorithms have been proposed that are supervised by different types of target content, such as 2.5D RGB-depth maps, 3D focal stacks, and 4D light fields. It is unclear, however, what the perceptual implications are of the choice of algorithm and target content type. In this work, we build a perceptual testbed of a full-color, high-quality holographic near-eye display. Under natural viewing conditions, we examine the effects of various CGH supervision formats and conduct user studies to assess their perceptual impacts on 3D realism. Our results indicate that CGH algorithms designed for specific viewpoints exhibit noticeable deficiencies in achieving 3D realism. In contrast, holograms incorporating parallax cues consistently outperform other formats across different viewing conditions, including the center of the eyebox. This finding is particularly interesting and suggests that the inclusion of parallax cues in CGH rendering plays a crucial role in enhancing the overall quality of the holographic experience. This work represents an initial stride towards delivering a perceptually realistic 3D experience with holographic near-eye displays.

FILES

CITATION

D. Kim*, S.-W. Nam*, S. Choi*, J.-M. Seo, G. Wetzstein, and Y. Jeong, “Holographic Parallax Improves 3D Perceptual Realism”, SIGGRAPH 2024.

@article{kim2024holographic,
title={Holographic Parallax Improves 3D Perceptual Realism},
author={Kim, Dongyeon and Nam, Seung-Woo and Choi, Suyeon and Seo, Jong-Mo and Wetzstein, Gordon and Jeong, Yoonchan},
journal={ACM Transactions on Graphics (TOG)},
volume={43},
number={4},
articleno = {68},
pages={1–13},
year={2024},
publisher={ACM New York, NY, USA}
}

Related Projects

You may also be interested in related projects from our group on holographic near-eye displays:

  • J. Kim et al. “Holographic Glasses”, SIGGRAPH 2022 (link)
  • S. Choi et al. “Time-multiplexed Neural Holography”, SIGGRAPH 2022 (link)
  • S. Choi et al. “Neural 3D Holography”, SIGGRAPH Asia 2021 (link)
  • Y. Peng et al. “Neural Holography”, SIGGRAPH Asia 2020 (link)
  • N. Padmanaban et al. “Holographic Near-Eye Displays Based on Overlap-Add Stereograms”, SIGGRAPH Asia 2019 (link)
Experimentally captured results on our holographic perceptual testbed with a moving camera. Light-field CGH optimization (denoted as 4D) produces more robust holographic imagery for non-ideal viewing points than image-based CGH algorithms.
3D holographic perceptual testbed and user study session. With recent advancements in computer-generated hologram (CGH) algorithms, the image quality of holographic displays has surpassed the threshold required for conducting robust user studies, enabling us to investigate the perceptual impact brought by modern holographic displays. We conduct user studies using the apparatus shown in this picture, using holographic scenes generated using various CGH methods.
User experiment results. 3D realism is assessed using CGHs supervised with four target formats (2.5D in yellow, 3D w/ RGB-D in green, 3D w/ LF in blue, 4D in red) across four viewing conditions (Center, De-centered, Vignetted, and with head movement). The mean JOD is set as zero for each viewing condition. Error bars represent 95% confidence intervals estimated by bootstrapping 500 samples. Asterisks (blue: 3D w/ LF vs. paired cases, red: 4D vs. other cases) indicate the statistical significance of differences (*: 𝑝<0.05, **: 𝑝<0.01, ***: 𝑝<0.001)
Reconstructed images with different pupil apodization profiles of the human eye. The prevalent CGH pipeline, reliant on naive low-pass filtering, overlooks the complexity of the human visual system. This leads to no variance in the retinal image despite eye movements when eye-box support is minimal. Our research motivates a compelling case for incorporating aspects of human vision, such as the directional sensitivity of the retina, into the rendering process to enhance the perceptual realism of holographic displays.