The human visual system is remarkably tolerant to degradation in image resolution: human performance in scene categorization remains high no matter whether low-resolution images or multimegapixel images are used. This observation raises the question of how many pixels are required to form a meaningful representation of an image and identify the objects it contains. In this article, we show that very small thumbnail images at the spatial resolution of 32 × 32 color pixels provide enough information to identify the semantic category of real-world scenes. Most strikingly, this low resolution permits observers to report, with 80% accuracy, four to five of the objects that the scene contains, despite the fact that some of these objects are unrecognizable in isolation. The robustness of the information available at very low resolution for describing semantic content of natural images could be an important asset to explain the speed and efficiently at which the human brain comprehends the gist of visual scenes.

During visual fixation, small eye movements keep the retinal image continuously in motion. It is known that neurons in the visual system are sensitive to the spatiotemporal modulations of luminance resulting from this motion. In this study, we examined the influence of fixational eye movements on the statistics of neural activity in the macaque's retina during the brief intersaccadic periods of natural visual fixation. The responses of parvocellular (P) and magnocellular (M) ganglion cells in different regions of the visual field were modeled while their receptive fields scanned natural images following recorded traces of eye movements. Immediately after the onset of fixation, wide ensembles of coactive ganglion cells extended over several degrees of visual angle, both in the central and peripheral regions of the visual field. Following this initial pattern of activity, the covariance between the responses of pairs of P and M cells and the correlation between the responses of pairs of M cells dropped drastically during the course of fixation. Cell responses were completely uncorrelated by the end of a typical 300-ms fixation. This dynamic spatial decorrelation of retinal activity is a robust phenomenon independent of the specifics of the model. We show that it originates from the interaction of three factors: the statistics of natural scenes, the small amplitudes of fixational eye movements, and the temporal sensitivities of ganglion cells. These results support the hypothesis that fixational eye movements, by shaping the statistics of retinal activity, are an integral component of early visual representations.

Assigning a basic color name to an object and rating the amount of a particular hue is a fundamental visual capability. Traditional color scaling studies have used increment flashes or isoluminant stimuli of a homogeneous color. Natural objects, however, do not contain a single color but are characterized by a distribution of different chromatic hues. Here we study color scaling using photographs of natural fruit objects. Stimuli were either homogeneous spots, digital photographs of fruit objects (e.g., banana), or outline shapes of the fruit objects. Stimuli were displayed on a CRT monitor on a homogeneous white background; its luminance was varied above and below the medium gray. The chromaticity of the stimuli was varied in 36 equally spaced chromatic directions in the isoluminant plane of the Derrington-Krauskopf-Lennie (DKL) color space. For each stimuli, subjects rated the amount of red, green, blue, and yellow in the stimulus on a scale from 0–8. In agreement with earlier studies we found that the positions of the peak ratings for each color do not coincide with the cardinal axis of DKL color space and are largely invariant under changes of the background luminance. For the average rating we found a dependence on background luminance for all colors: yellow ratings increase with darker backgrounds, whereas ratings for the other colors, in particular green, decrease. For the fruit objects, we found a selective increase in the average color rating for the natural fruit color. For example, the average rating for yellow was 1.7 times higher for the banana images compared to disc stimuli. No such selective increase was found for outline shapes. We conclude that the distribution of hues in natural objects with a characteristic object color can have a profound effect on color scaling and color appearance.

Intracellular recordings were obtained from 57 cone-driven bipolar cells in the light-adapted retina of the land-phase (adult) tiger salamander (Ambystoma tigrinum). Responses to flashes of negative and positive contrast for centered spots of optimum spatial dimensions were analyzed as a function of contrast magnitude. On average, the contrast/response curves of depolarizing and hyperpolarizing bipolar cells in the land-phase animals were remarkably similar to those of aquatic-phase animals. Thus, the primary retinal mechanisms mediating contrast coding in the outer retina are conserved as the salamander evolves from the aquatic to the land phase. To evaluate contrast encoding in the context of natural environments, the distribution of contrasts in natural images was measured for 65 scenes. The results, in general agreement with other reports, show that the vast majority of contrasts in nature are very small. The efficient coding hypothesis of Laughlin was examined by comparing the average contrast/response curves of bipolar cells with the cumulative probability distribution of contrasts in natural images. Efficient coding was found at 20 cd/m2 but at lower levels of light adaptation, the contrast/response curves were much too shallow. Further experiments show that two fundamental physiological factors—light adaptation and the nonlinear transfer across the cone-bipolar synapse are essential for the emergence of efficient contrast coding. For both land- and aquatic-based animals, the extent and symmetry of the dynamic range of the contrast/response curves of both classes of bipolar cells varied greatly from cell to cell. This apparent substrate for distributed encoding is established at the bipolar cell level, since it is not found in cones. As a result, the dynamic range of the bipolar cell population brackets the distribution of contrasts found in natural images.

Lateral inhibition is one of the first and most important stages of visual processing. There are at least four theories related to information theory in the literature for the role of early retinal lateral inhibition. They are based on the spatial redundancy in natural images and the advantage of removing this redundancy from the visual code. Here, we contrast these theories with data from the retina's outer plexiform layer. The horizontal cells' lateral-inhibition extent displays a bell-shape behavior as function of background luminance, whereas all the theories show a fall as luminance increases. It is remarkable that different theories predict the same luminance behavior, explaining “half” of the biological data. We argue that the main reason is how these theories deal with photon-absorption noise. At dim light levels, for which this noise is relatively large, large receptive fields would increase the signal-to-noise ratio through averaging. Unfortunately, such an increase at low luminance levels may smooth out basic visual information of natural images. To explain the biological behavior, we describe an alternate hypothesis, which proposes that the role of early visual lateral inhibition is to deal with noise without missing relevant clues from the visual world, most prominently, the occlusion boundaries between objects.