Adaptive Spectral Projection | SIGGRAPH Asia 2015

Isaac Kauvar, Samuel J. Yang, Liang Shi, Ian McDowall, Gordon Wetzstein

A computational projection system that provides content-adaptive color primaries, which are optimized in a perceptually-optimal manner.

 

Abstract

Fundamental display characteristics are constantly being improved, especially resolution, dynamic range, and color reproduction. However, whereas high resolution and high-dynamic range displays have matured as a technology, it remains largely unclear how to extend the color gamut of a display without either sacrificing light throughput or making other tradeoffs. In this paper, we advocate for adaptive color display; with hardware implementations that allow for color primaries to be dynamicallychosen, an optimal gamut and corresponding pixel states can be computed in a content-adaptive and user-centric manner. We build a flexible gamut projector and develop a perceptually-driven optimization framework that robustly factors a wide color gamut target image into a set of time-multiplexed primaries and corresponding pixel values. We demonstrate that adaptive primary selection has many benefits over fixed gamut selection and show that our algorithm for joint primary selection and gamut mapping performs better than existing methods. Finally, we evaluate the proposed computational display system extensively in simulation and, via photographs and user experiments, with a prototype adaptive color projector.

FILES

  • technical paperĀ (pdf)
  • technical paper supplement (pdf)
  • presentation slides (slideshare)

 

CITATION

I. Kauvar, S. Yang, L. Shi, I. McDowall, G. Wetzstein. “Adaptive Color Display via Perceptually-driven Factored Spectral Projection”, ACM SIGGRAPH Asia (Transactions on Graphics), 2015.

BibTeX

@article{Kauvar:2015:AdaptiveSpectralProjection,
author = {I. Kauvar and S. Yang and L. Shi and I. McDowall and G. Wetzstein},
title = {{Adaptive Color Display via Perceptually-driven Factored Spectral Projection}},
journal = {ACM Trans. Graph. (SIGGRAPH Asia)},
year = {2015},
}

ADDITIONAL MATERIAL

A summary of the project: (top left) hardware prototype of flexible gamut projector. (middle) prototype in action. (bottom) convergence of perceptual non-negative matrix factorization (PNMF) algorithm to final gamut selection and gamut mapping, shown here in CIEXYZ color space.

 

PNMF algorithm performs better than legacy algorithms. (column #1) recovered image, clipped to sRGB color gamut. (columns #2,3,4) error in CIELAB76 color space between recovered image and target multispectral image. PNMF yields the lowest error. (columns #5,6) final color gamuts determined by each algorithm, shown as 2D and 3D representations. (column #7) convergence of each algorithm.

 

Schematic of formulation as joint gamut selection and gamut mapping as a non-negative matrix factorization problem. Gamut selection corresponds to choosing the linear combination of LED intensities corresponding to each primary color for each time point; gamut mapping consists of choosing the grayscale value of each pixel on the DMD to be displayed at each time point corresponding to a primary color.

Related Projects

You may also be interested in related projects from our group on next-generation near-eye displays and wearable technology:

  • Y. Peng et al. “Neural Holography with Camera-in-the-loop Training”, ACM SIGGRAPH 2020 (link)
  • R. Konrad et al. “Gaze-contingent Ocular Parallax Rendering for Virtual Reality”, ACM Transactions on Graphics 2020 (link)
  • B. Krajancich et al. “Optimizing Depth Perception in Virtual and Augmented Reality through Gaze-contingent Stereo Rendering”, ACM SIGGRAPH Asia 2020 (link)
  • B. Krajancich et al. “Factored Occlusion: Single Spatial Light Modulator Occlusion-capable Optical See-through Augmented Reality Display”, IEEE TVCG, 2020 (link)
  • N. Padmanaban et al. “Autofocals: Evaluating Gaze-Contingent Eyeglasses for Presbyopes”, Science Advances 2019 (link)
  • K. Rathinavel et al. “Varifocal Occlusion-Capable Optical See-through Augmented Reality Display based on Focus-tunable Optics”, IEEE TVCG 2019 (link)
  • N. Padmanaban et al. “Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays”, PNAS 2017 (link)
  • R. Konrad et al. “Accommodation-invariant Computational Near-eye Displays”, ACM SIGGRAPH 2017 (link)
  • R. Konrad et al. “Novel Optical Configurations for Virtual Reality: Evaluating User Preference and Performance with Focus-tunable and Monovision Near-eye Displays”, ACM SIGCHI 2016 (link)
  • F.C. Huang et al. “The Light Field Stereoscope: Immersive Computer Graphics via Factored Near-Eye Light Field Display with Focus Cues”, ACM SIGGRAPH 2015 (link)