Figure 1: Tone reproduction operator.
Ferwerda, J.A., Pattanaik, S., Shirley, P., and Greenberg, D.P. (1996)
A
model of visual adaptation for realistic image synthesis. Proceedings
SIGGRAPH '96, 249-258.
Abstract PDF
(635Kb)
Pattanaik, S., Ferwerda, J.A., Fairchild, M.D., and Greenberg, D.P.
(1998) A multiscale model of adaptation and spatial vision for realistic
image display. Proceedings SIGGRAPH '98, 287-298.
Abstract PDF
(1.5Mb)
The range of light energy we encounter in natural scenes is vast. The absolute level of illumination in a scene can vary dramatically from day to night, with direct sunlight being 100 million times more intense than starlight. The dynamic range of light energy in a scene can also be large, with variations on the order of 10,000 to 1 from highlights to shadows. Unfortunately both the absolute and dynamic ranges of existing display devices are small, with only moderate absolute output levels, and useful dynamic ranges less than 100 to 1. Given this discrepancy between the wide ranges we encounter in the environment and the small ranges that can be reproduced by existing displays, in digital imaging we are faced with the tone reproduction problem: how should we map from scene intensities to display intensities to produce an image that captures the appearance of the scene?
Recently graphics researchers have started to address this issue by developing tone reproduction operators that map scene intensities to display intensities with the goal of producing a visual match between the scene and the display.
Figure 1: Tone reproduction operator.
The major components of a typical tone reproduction operator are illustrated in Figure 1. On the left side of the figure are the input scene radiances. A hypothetical scene observer receives these radiant intensities and has a particular visual experience. On the right side of the diagram are the inputs to a display device viewed by a display observer. The display observer looks at the image on the display and also has a particular visual experience. The goal of realistic tone reproduction is to have the display observer have the same visual experience as the scene observer, producing a visual match between the scene and the display. To achieve this, a tone reproduction operator is introduced that maps from scene intensities to display intensities with the goal of producing an image that captures the appearance of the scene. There are two major problems to be solved in realistic tone reproduction:
The critical element that links these two problems is the visual model used in the tone reproduction operator. Visual models are used to relate the perceptual responses of the scene observer to the responses of the display observer to produce a displayed image that realistically captures the appearance of the scene.
Since 1996, we have been developing new visual models for realistic tone reproduction in digital imaging. Our models are based on the psychophysics of luminance, pattern, and color processing in the human visual system and simulate the changes in brightness, apparent contrast, color appearance, and visual acuity that occur with changes in the level of illumination in a scene.
We have incorporated the models into tone reproduction operators that map the vast ranges of intensities found in real and synthetic scenes into the small fixed ranges available on conventional display devices such as CRT's and printers. These new operators address the two major problems in realistic tone reproduction: images of wide absolute range and high dynamic range scenes can be displayed; and the displayed images capture the appearance of the scenes at both threshold and suprathreshold levels.
Figure 2: Displaying wide absolute range scenes [Best viewed with display gamma set to 2.2].
Figure 2 shows our '96 operator applied to a wide absolute range scene. The image on the left shows the scene illuminated at a daylight level of 1000 cd/m2. Notice that visibility is good, all the colors are fully saturated, and acuity is high because even the smallest characters in the Snellen eye chart can be identified. The image on the right shows the scene illuminated at a nighttime, moonlight level of 0.04 cd/m2. Notice here that due to visual adaptation, visibility is reduced, colors have lost their saturation, and acuity is poor.
Figure 3: Displaying high dynamic range scenes [Best viewed with display gamma set to 2.2].
Figure 3 shows our '98 operator applied to a high dynamic range scene. This scene has about a 10,000-to-1 dynamic range from exterior highlights to interior shadows. The image on the left shows the result produced by linearly mapping the this large range into the narrow 50-to-1 range of the display. Notice that the linear mapping loses significant amounts of detail at both high and low levels of illumination. The image on the right shows the result produced by applying our tone reproduction operator. This visual mapping allows scene features to be seen across the intensity range, and better reproduces the scene's actual appearance.
The images produced capture the dramatic changes in visual appearance produced by changes in the level of illumination in natural and simulated scenes. Because the oprators are based on psychophysical data they can be used to generate predictive visual simulations that show a display viewer what they would and would not be able to see if they were actually standing in a scene.
Finally, beyond the clear application of this work to the problem of realistic tone reproduction, the visual models we are developing are general and can be usefully applied to a variety of other important problems in digital imaging including: image quality metrics; image coding and compression methods; perceptually-based rendering; and advanced display system design.