Skip to main content Accessibility help

Surface color perception in three-dimensional scenes



Researchers studying surface color perception have typically used stimuli that consist of a small number of matte patches (real or simulated) embedded in a plane perpendicular to the line of sight (a “Mondrian,” Land & McCann, 1971). Reliable estimation of the color of a matte surface is a difficult if not impossible computational problem in such limited scenes (Maloney, 1999). In more realistic, three-dimensional scenes the difficulty of the problem increases, in part, because the effective illumination incident on the surface (the light field) now depends on surface orientation and location. We review recent work in multiple laboratories that examines (1) the degree to which the human visual system discounts the light field in judging matte surface lightness and color and (2) what illuminant cues the visual system uses in estimating the flow of light in a scene.


Corresponding author

Address correspondence and reprint requests to: Huseyin Boyacı, University of Minnesota at Twin Cities, Department of Psychology, 75 East River Rd., 219, Minneapolis, MN 55455, USA. E-mail:


Hide All


Adelson, E.H. & Bergen, J.R. (1991). The plenoptic function and the elements of early vision. In Computational Models of Visual Processing, eds. Landy, M.S. & Movshon, J.A., pp. 320. Cambridge, MA: MIT Press.
Adelson, E.H. & Pentland, A.P. (1996). The perception of shading and reflectance. In Perception as Bayesian Inference, eds. Knill, D. & Richards, W., pp. 409423. New York: Cambridge University Press.
Arend, L.E. & Spehar, B. (1993a). Lightness, brightness, and brightness contrast: 1. illuminance variation. Perception & Psychophysics 54, 446456.
Arend, L.E. & Spehar, B. (1993b). Lightness, brightness, and brightness contrast: 2. reflectance variation. Perception & Psychophysics 54, 457468.
Bäuml, K.-H. (1999). Simultaneous color constancy: how surface color perception varies with the illuminant. Vision Research 39, 15311550.
Belhumeur, P.N., Kriegman, D., & Yuille, A. (1999). The bas-relief ambiguity. International Journal of Computer Vision 35(1), 3344.
Bloj, M.G., Kersten, D., & Hurlbert, A.C. (1999). Perception of three-dimensional shape influences colour perception through mutual illumination. Nature 402, 877879.
Boyaci, H., Doerschner, K., & Maloney, L.T. (2004). Perceived surface color in binocularly-viewed scenes with two light sources differing in chromaticity. Journal of Vision 4, 664679.
Boyaci, H., Doerschner, K., & Maloney, L.T. (2006). Cues to an equivalent lighting model, Journal of Vision 6, 106118.
Boyaci, H., Maloney, L.T., & Hersh, S. (2003). The effect of perceived surface orientation on perceived surface albedo in three-dimensional scenes, Journal of Vision 3, 541553.
Ciurea, F. & Funt, B. (2004). Failure of luminance-redness correlation for illuminant estimation. Proceedings Twelfth Color Imaging Conference, pp. 4246.
Doerschner, K., Boyaci, H., & Maloney, L.T. (2004). Human observers compensate for secondary illumination originating in nearby chromatic surfaces, Journal of Vision 4, 92105.
Dror, R.O., Willsky, A., & Adelson, E.H. (2004). Statistical characterization of real-world illumination. Journal of Vision 4, 821837.
Epstein, W. (1961). Phenomenal orientation and perceived achromatic color. Journal of Psychology 52, 5153.
Fleming, R.W., Dror, R.O., & Adelson, E.H. (2003). Real-world illumination and the perception of surface reflectance properties. Journal of Vision 3, 347368.
Flock, H.R. & Freedberg, E. (1970). Perceived angle of incidence and achromatic surface color. Perception & Psychophysics 8, 251256.
Foster, D.H. & Nascimento, S.M.C. (1994). Relational colour constancy from invariant cone-excitation ratios. Proceedings of the Royal Society of London B 257, 115121.
Gershun, A. (1936/1939). Svetovoe Pole (English: The Light Field), Moscow, 1936. Translated by P. Moon and G. Timoshenko (1939) in Journal of Mathematics and Physics 18, 51151.
Gilchrist, A.L. (1977). Perceived lightness depends on spatial arrangement. Science 195, 185187.
Gilchrist, A.L. (1980). When does perceived lightness depend on perceived spatial arrangement? Perception & Psychophysics 28, 527538.
Gilchrist, A.L. & Annan, Jr., A. (2002). Articulation effects in lightness: historical background and theoretical implications. Perception 31, 141150.
Gilchrist, A.L., Kossyfidis, C., Bonato, F., Agostini, T., Cataliotti, J., Li, X.J., Spehar, B., Annan, V., & Economou, E. (1999). An anchoring theory of lightness perception. Psychological Review 106, 795834.
Golz, J. & MacLeod, D.I.A. (2002). Influence of scene statistics on colour constancy. Nature 415, 637640.
Granzier, J.J.M., Brenner, E., Cornelissen, F.W., & Smeets, J.B.J. (2005). Luminance–color correlation is not used to estimate the color of the illumination. Journal of Vision 5, 2027.
Hara, K., Nishino, K., & Ikeuchi, K. (2005). Light source position and reflectance estimation from a single view without the distant illumination assumption. IEEE Transactions on Pattern and Machine Intelligence 27, 493505
Haralick, R.M. & Shapiro, L.G. (1993). Computer and robot vision, Vol. 2. Reading, MA: Addison-Wesley.
Henderson, S.T. (1977). Daylight and its spectrum, 2nd Ed. Bristol, UK: Adam Hilger.
Hochberg, J.E. & Beck, J. (1954). Apparent spatial arrangements and perceived brightness. Journal of Experimental Psychology 47, 263266.
Hurlbert, A.C. (1998). Computational models of colour constancy. In Perceptual Constancy: Why things look as they do, eds. Walsh, V. & Kulikowski, J., pp. 283322. Cambridge, UK: Cambridge University Press.
Ikeda, M., Shinoda, H., & Mizokami, Y. (1998). Three dimensionality of the recognized visual space of illumination proved by hidden illumination. Optical Review 5, 200205.
Kaiser, P.K. & Boynton, R.M. (1996). Human color vision, 2nd Ed. Washington, DC: Optical Society of America.
Kardos, L. (1934). Ding und Schatten; Eine experimentelle Untersuchung über die Grundlagen des Farbsehens, Zeitschrift für Psychologie and Physiologie der Sinnesorgane, Ergänzungsband 23, Leipzig, Germany: Verlag von J.A. Barth. (Edited by Schumann, F., Jaensch, E.R. & Kroh, O.).
Katz, D. (1935). The World of Colour. London: Kegan, Paul, Trench, Trubner & Co.
Koenderink, J.J. & van Doorn, A.J. (1996). Illuminance texture due to surface mesostructure. Journal of the Optical Society of America A 13, 452463.
Koenderink, J.J., van Doorn, A.J., & Pont, S.C. (2004). Light direction from shad(ow)ed random Gaussian surfaces. Perception 33 (12), 14031404, special issue: Shadows and Illumination II.
Land, E.H. & McCann, J.J. (1971). Lightness and retinex theory. Journal of the Optical Society of America 61, 111.
Lee, Jr., R.L. & Hernández-Andrés, J. (2005a). Short-term variability of overcast brightness. Applied Optics 44, 57045711.
Lee, Jr., R.L. & Hernández-Andrés, J. (2005b). Colors of the daytime overcast sky. Applied Optics 44, 57125722.
MacLeod, D.I.A. & Golz, J. (2003). A computational analysis of colour constancy. In Colour Perception: Mind and the Physical World, eds. Mausfeld, R. & Heyer, D., pp. 205242. Oxford, UK: Oxford University Press.
Maloney, L.T. (1999). Physics-based approaches to modeling surface color perception. In Color Vision: From Genes to Perception, eds. Gegenfurtner, K.R. & Sharpe, L.T., pp. 387422. Cambridge, UK: Cambridge University Press.
Maloney, L.T., Boyaci, H., & Doerschner, K. (2005). Surface color perception as an inverse problem in biological vision. Proceedings of the SPIE-IS & T Electronic Imaging 5674, 1526.
Mausfeld, R. & Andres, J. (2002). Second-order statistics of colour codes modulate transformations that effectuate varying degrees of scene invariance and illumination invariance. Perception 31, 209224.
Nascimento, S.M.C. & Foster, D.H. (2000). Relational color constancy in achromatic and isoluminant images. Journal of the Optical Society of America A-Optics Image Science and Vision 17, 225231.
Pont, S.C. & Koenderink, J.J. (2003). Illuminance flow. In Computer Analysis of Images and Patterns, eds. Petkov, N. & Wetsenberg, M.A., pp. 9097. Berlin: Springer-Verlag.
Pont, S.C. & Koenderink, J.J. (2004). Surface illuminance flow. Proceedings Second International Symposium on 3D Data Processing Visualization and Transmission, eds. Aloimonos, Y. & Taubin, G.
Redding, G.M. & Lester, C.F. (1980). Achromatic color matching as a function of apparent test orientation, test and background luminance, and lightness or brightness instructions. Perception & Psychophysics 27, 557563.
Ripamonti, C., Bloj, M., Hauck, R., Kiran, K., Greenwald, S., Maloney, S.I., & Brainard, D.H. (2004). Measurements of the effect of surface slant on perceived lightness. Journal of Vision 4, 747763.
Snyder, J.L., Doerschner, K., & Maloney, L.T. (2005). Illumination estimation in three-dimensional scenes with and without specular cues. Journal of Vision 5, 863877.
te Pas, S.F. & Pont, S.C. (2005). Comparison of material and illumination discrimination performance for real rough, real smooth and computer generated smooth spheres. In Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization (A Coroña, Spain, August 26–28, 2005). APGV '05, vol. 95. ACM Press, New York, NY, 7581.
Yang, J.N. & Maloney, L.T. (2001). Illuminant cues in surface color perception: Tests of three candidate cues. Vision Research 41, 25812600.



Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed