It is generally accepted that there is something special about reasoning by using mental images. The question of how it is special, however, has never been satisfactorily spelled out, despite more than thirty years of research in the post-behaviorist tradition. This article considers some of the general motivation for the assumption that entertaining mental images involves inspecting a picture-like object. It sets out a distinction between phenomena attributable to the nature of mind to what is called the cognitive architecture, and ones that are attributable to tacit knowledge used to simulate what would happen in a visual situation. With this distinction in mind, the paper then considers in detail the widely held assumption that in some important sense images are spatially displayed or are depictive, and that examining images uses the same mechanisms that are deployed in visual perception. I argue that the assumption of the spatial or depictive nature of images is only explanatory if taken literally, as a claim about how images are physically instantiated in the brain, and that the literal view fails for a number of empirical reasons – for example, because of the cognitive penetrability of the phenomena cited in its favor. Similarly, while it is arguably the case that imagery and vision involve some of the same mechanisms, this tells us very little about the nature of mental imagery and does not support claims about the pictorial nature of mental images. Finally, I consider whether recent neuroscience evidence clarifies the debate over the nature of mental images. I claim that when such questions as whether images are depictive or spatial are formulated more clearly, the evidence does not provide support for the picture-theory over a symbol-structure theory of mental imagery. Even if all the empirical claims were true, they do not warrant the conclusion that many people have drawn from them: that mental images are depictive or are displayed in some (possibly cortical) space. Such a conclusion is incompatible with what is known about how images function in thought. We are then left with the provisional counterintuitive conclusion that the available evidence does not support rejection of what I call the “null hypothesis”; namely, that reasoning with mental images involves the same form of representation and the same processes as that of reasoning in general, except that the content or subject matter of thoughts experienced as images includes information about how things would look.

According to Pylyshyn, depictive representations can be explanatory only if a certain kind of first-order isomorphism exists between the mental representations and real-world displays. What about a system with second-order isomorphism (similarities between different mental representations corresponding with similarities between different real-world displays)? Such a system may help to address whether “depictive” representations contribute to the visual nature of imagery.

The “Perky effect” is the interference of visual imagery with vision. Studies of this effect show that visual imagery has more than symbolic properties, but these properties differ both spatially (including “pictorially”) and temporally from those of vision. We therefore reject both the literal picture-in-the-head view and the entirely symbolic view.

Right brain-damaged patients with unilateral neglect, who ignore left-sided visual events, may also omit left-sided details when describing known places from memory. Modulating the orienting of visual attention may ameliorate imaginal neglect. A first step toward explaining these phenomena might be to postulate that space-related imagery is a cognitive activity involving attentional and intentional aspects.

Hemispatial neglect in imagery implies a spatially organised representation. Reaction times in memory for arrays of locations from shifted viewpoints indicate processes analogous to actual bodily movement through space. Behavioral data indicate a privileged role for this process in memory. A proposed spatial mechanism makes contact with direct recordings of the representations of location and orientation in the mammalian brain.

Data from neuropsychology do not support the idea that the primary visual cortex necessarily displays internal visual images. However, the choice of formats used in human cognition is not restricted to depictive or descriptive representations. Nestled between pictures and propositions, primitive spatial schemas with simple analog features extracted from pictorial scenes may play a subtle but wide role in cognition.

Is mental imagery pictorial? In Pylyshyn's view no empirical data provides convincing support to the “pictorial” hypothesis of mental imagery. Phenomenology, Pylyshyn says, is deeply deceiving and offers no explanation of why and how mental imagery occurs. We suggest that Pylyshyn mistakes phenomenology for what it never pretended to be. Phenomenological evidence, if properly considered, shows that mental imagery may indeed be pictorial, though not in the way that mimics visual perception. Moreover, Pylyshyn claims that the “pictorial hypothesis” is flawed because the interpretation of “picture-like” objects in mental imagery takes a homunculus. However, the same point can be objected to Pylyshyn's own conclusion: if imagistic reasoning involves the same mechanisms and the same forms of representation as those that are involved in general reasoning, if they operate on symbol-based representations of the kind recommended by Pylyshyn (1984) and Fodor (1975), don't we need a phenomenological homunculus to tell an imagined bear from the real one?

We argue that the field has moved forward from the old debate about “analogical” versus “symbolic” processing. First, it is questionable that there is a strong a priori argument for assuming a common processing mode. Second, we explore the possibility that imagery is not a unitary mental function. Finally, we discuss the empirical basis of the involvement of primary areas.

The presence of spatial patterns of activity in the brain is suggestive of image-exploiting processes in vision and mental imagery, but not conclusive. Only behavioral evidence can confirm or disconfirm hypotheses about whether, and how, the brain uses images in its information-processing, and the arguments based on such evidence are still inconclusive.

Pylyshyn rightly argues that the neuroscientific data supporting the involvement of the visual system in mental imagery is largely irrelevant to the question of the format of imagistic representation. The purpose of this commentary is to support this claim with a further argument.

Observations on patients who lost visual imagery after brain damage call into question the notion that the knowledge subserving visual imagery is “tacit.” Dissociations between deficient imagery and preserved recognition of objects suggest that imagery is exclusively based on explicit knowledge, whereas retrieval of “tacit” visual knowledge is bound to the presence of the object and the task of recognizing it.

Pylyshyn argues against representations with pictorial properties that would be superimposed on a scene. We present evidence against this view, and a new method to depict pictorial properties. We propose a continuum between the top-down generation of internal signals (imagery) and the bottom-up signals from the outside world. Along the continuum, superstitious perceptions provide a method to tackle representational issues.

The descriptive “null” hypothesis is strengthened by the fact that during dreaming sleep stage, the primary visual cortex is deactivated as compared with other sleep stages.

Pylyshyn raises hot topics like the number and kinds of pictorialist theories there are and their explanatory power. Pylyshyn states that pictorialists have only two possibilities – they can posit either “only functional” images or “really spatial” images – and that neither of these possibilities is convincing or sufficient in explanatory power for empirical and theoretical reasons. Is pictorialism, in principle, untenable?

Recent neural models clarify many properties of mental imagery as part of the process whereby bottom-up visual information is influenced by top-down expectations, and how these expectations control visual attention. Volitional signals can transform modulatory top-down signals into supra-threshold imagery. Visual hallucinations can occur when the normal control of these volitional signals is lost.

I support Pylyshyn's skepticism that visual imagery reflects a re-activation of the spatial layout of active neurons embedded within a topographical cortical map of visual space. The pickup of visual information via successive eye movements presents one problem and the two visual systems model poses another difficulty.

This commentary raises questions about the central concepts in the null hypothesis presented by the author of the target article and urges expansion of the treatment of mental imagery to forms of sensory imagery beyond the visual.

Spatial objects may not only be perceived visually but also by touch. We report recent experiments investigating to what extent prior object knowledge acquired in either the haptic or visual sensory modality transfers to a subsequent visual learning task. Results indicate that even mental object representations learnt in one sensory modality may attain a multi-modal quality. These findings seem incompatible with picture-based reasoning schemas but leave open the possibility of modality-specific reasoning mechanisms.

This commentary focuses on four major points: (1) “Tacit knowledge” is not a complete explanation for imagery phenomena, if it is an explanation at all. (2) Similarities and dissimilarities between imagery and perception are entirely consistent with the depictive view. (3) Knowledge about the brain is crucial for settling the debate. (4) It is not clear what sort of theory Pylyshyn advocates.

Various techniques have attempted to localize imagery. However, early findings using single cell recordings of human receptive fields during imagery tasks have had little impact. Reports by Marg and his coworkers (1968) found no evidence for imagery in human Area 17, 18, and 19. Single cells from humans suggest later imagery-related activity in hippocampus, amygdala, entorhinal cortex, and parahippocampal gyrus.