In their wonderful and provocative target article, Quilty-Dunn et al. say perceptual object representations have language-of-thought (LoT) structure. However, there is plenty of evidence that perceptual object representations are iconic in a sense that excludes LoT representations; the evidence Quilty-Dunn et al. cite pertains to discursive perceptual working memory (WM) representations, not discursive perceptual representations. I will first present some evidence that perceptual object representations are iconic, then that WM representations are discursive. I will use the term “discursive” for representations that exhibit almost all of the six properties they cite rather than “LoT” because I doubt that even perceptual working memory exhibits all of them. (See Susan Carey's response to the target article.) But if there are no discursive representations in perception, there are no LoT representations either.
Apparent motion suggests iconic perceptual object representations. When two nearby objects flicker with the right parameters, we see motion between them. Objects move while visible properties change gradually.
See the caption to Figure 1. What suggests iconicity in this case of apparent motion is analog mirroring: Certain relations in the world are mirrored by representations that instantiate analogs of those relations in a way that is sensitive to degrees of difference. What is interesting about this case is that when the figure is perceived as an object, mirroring respects objecthood.
Figure 1. Items at Y1 and Y2 flicker so as to create apparent motion between them. The shapes are viewed in an apparatus in which a slightly different image is projected to each eye. This allows one version in which the black bar is part of a squarish shape and another version in which the bar protrudes, making the item look like an object instead of a shape. In the latter case, the subject sees rotation along with the movement but in the former case the subject sees just motion. Thanks to Ken Nakayama for providing this figure.
If the flickering objects are of different sizes, we see smooth expansion and contraction. One might suppose that the further apart the flickering objects are, the faster the rate of flicker would have to be to see motion. However, mirroring dictates the opposite because objects that are further apart take longer to traverse the distance. The further apart the flickering objects are, the longer the time span between flickers has to be to see motion (Korte's Third Law). The visual system prefers short motion paths between flickering objects but that preference is overridden if the shortest path involves biologically impossible motion (Shiffrar & Freyd, Reference Shiffrar and Freyd1990) or if a moving object turns into an object of a different kind. In sum, perceptual representations are iconic in a sense that excludes discursive representations (see Block, Reference Block2023a, Reference Block, McLaughlin and Cohen2023b).
I now switch to the topic of perceptual working memory. Perceptual working memory often contains the remnants of perception – typically not consciously experienced. It can include iconic materials, but visual working memory often includes them in recoded form. One illustration of the partially nonperceptual nature of visual working memory is illustrated in Figure 2. When the central disk and the donut surrounding it are presented simultaneously, there is center-surround suppression on the right, but not the left. However, when they are presented one at a time, with the first stimulus maintained in working memory, the collinear effect disappears (Bloem, Watanabe, Kibbe, & Ling, Reference Bloem, Watanabe, Kibbe and Ling2018). Thus a fundamental computational aspect of perception is absent in this working memory representation (see pp. 113–114 of Block, Reference Block2023a).
Figure 2. Central disk is the same on the left and on the right but it looks much higher in contrast on the left. The suppressive center/surround effect in the collinear case is because of a fundamental computation known as divisive normalization. Thanks to Sam Ling for providing this figure.
Quilty-Dunn et al.'s arguments for the iconic nature of perception involve the “object files” of perceptual working memory. But working memory representations in visual areas are often recoded outside of the classic visual system, for example, in the intraparietal sulcus, while they disappear from visual cortex because of ongoing visual stimulation (Rademaker, Chunharas, & Serences, Reference Rademaker, Chunharas and Serences2019). Perceptual information is often recoded in the service of specific tasks. Kwak and Curtis (Reference Kwak and Curtis2022) showed subject clouds of moving dots and also oriented gratings, asking them to remember the directions. They found that brain decoding on either of these working memory representation worked on the other suggesting that working memory coded what was in common to the two kinds of percepts, eliminating the moving dots and the gratings, replacing them with representations of vectors, showing that many iconic features can be altered or discarded in working memory.
Of course perceptual working memory is constantly interacting with perception. Quilty-Dunn (Reference Quilty-Dunn2023) argues convincingly that this interaction is crucial to longer term perceptions, for example, perceptions that span saccades. As Quilty-Dunn notes, perception does not start anew after each saccade, so there must be some perceptual – I would say iconic – information preserved by the saccade. True, but there is plenty of evidence for at least some loss of iconicity in transsaccadic memory (summarized in Block, Reference Block2023a, pp. 261–262). For example, in the famous Sperling effect, a multirow array of letters is presented briefly, but a cue presented after the stimulus stops can focus attention on any one of the rows, allowing reporting of all or almost all the items. However, if the array is presented before the saccade and the cue presented afterward, the Sperling effect disappears, showing that transsaccadic memory can erase the iconic memory that the Sperling effect depends on.
Quilty-Dunn describes a perceptual effect known as the motion repulsion illusion. If dots moving in one direction are superimposed on dots moving in a different direction, the perceived angle between the two directions is exaggerated in perception. Kang et al. showed that the same effect occurs if one set of moving dots is seen while the other is held in working memory (Kang, Hong, Blake, & Woodman, Reference Kang, Hong, Blake and Woodman2011). This result suggests that there are iconic elements in perceptual working memory but does nothing to show that perception is not iconic.
As Quilty-Dunn notes, perception can distort working memory and conversely. Teng and Kravitz (Reference Teng and Kravitz2019) showed that colors and orientations in each of perception and working memory affect the other, commenting that this is no doubt because of overlapping representations in visual processing areas. However, as the authors note, these results are compatible with the involvement of prefrontal cortex in working memory. The overlap of sensory coding between visual working memory and vision does not preclude partial reformatting in working memory or the inclusion of iconic information in a discursive envelope.
In sum, perceptual working memory can preserve some aspects of iconic perceptual representation even if it includes it in a discursive envelope. Quilty-Dunn et al.'s results depend on the discursive envelope, not the iconic perceptual representations.
You have
Access
In their wonderful and provocative target article, Quilty-Dunn et al. say perceptual object representations have language-of-thought (LoT) structure. However, there is plenty of evidence that perceptual object representations are iconic in a sense that excludes LoT representations; the evidence Quilty-Dunn et al. cite pertains to discursive perceptual working memory (WM) representations, not discursive perceptual representations. I will first present some evidence that perceptual object representations are iconic, then that WM representations are discursive. I will use the term “discursive” for representations that exhibit almost all of the six properties they cite rather than “LoT” because I doubt that even perceptual working memory exhibits all of them. (See Susan Carey's response to the target article.) But if there are no discursive representations in perception, there are no LoT representations either.
Apparent motion suggests iconic perceptual object representations. When two nearby objects flicker with the right parameters, we see motion between them. Objects move while visible properties change gradually.
See the caption to Figure 1. What suggests iconicity in this case of apparent motion is analog mirroring: Certain relations in the world are mirrored by representations that instantiate analogs of those relations in a way that is sensitive to degrees of difference. What is interesting about this case is that when the figure is perceived as an object, mirroring respects objecthood.
Figure 1. Items at Y1 and Y2 flicker so as to create apparent motion between them. The shapes are viewed in an apparatus in which a slightly different image is projected to each eye. This allows one version in which the black bar is part of a squarish shape and another version in which the bar protrudes, making the item look like an object instead of a shape. In the latter case, the subject sees rotation along with the movement but in the former case the subject sees just motion. Thanks to Ken Nakayama for providing this figure.
If the flickering objects are of different sizes, we see smooth expansion and contraction. One might suppose that the further apart the flickering objects are, the faster the rate of flicker would have to be to see motion. However, mirroring dictates the opposite because objects that are further apart take longer to traverse the distance. The further apart the flickering objects are, the longer the time span between flickers has to be to see motion (Korte's Third Law). The visual system prefers short motion paths between flickering objects but that preference is overridden if the shortest path involves biologically impossible motion (Shiffrar & Freyd, Reference Shiffrar and Freyd1990) or if a moving object turns into an object of a different kind. In sum, perceptual representations are iconic in a sense that excludes discursive representations (see Block, Reference Block2023a, Reference Block, McLaughlin and Cohen2023b).
I now switch to the topic of perceptual working memory. Perceptual working memory often contains the remnants of perception – typically not consciously experienced. It can include iconic materials, but visual working memory often includes them in recoded form. One illustration of the partially nonperceptual nature of visual working memory is illustrated in Figure 2. When the central disk and the donut surrounding it are presented simultaneously, there is center-surround suppression on the right, but not the left. However, when they are presented one at a time, with the first stimulus maintained in working memory, the collinear effect disappears (Bloem, Watanabe, Kibbe, & Ling, Reference Bloem, Watanabe, Kibbe and Ling2018). Thus a fundamental computational aspect of perception is absent in this working memory representation (see pp. 113–114 of Block, Reference Block2023a).
Figure 2. Central disk is the same on the left and on the right but it looks much higher in contrast on the left. The suppressive center/surround effect in the collinear case is because of a fundamental computation known as divisive normalization. Thanks to Sam Ling for providing this figure.
Quilty-Dunn et al.'s arguments for the iconic nature of perception involve the “object files” of perceptual working memory. But working memory representations in visual areas are often recoded outside of the classic visual system, for example, in the intraparietal sulcus, while they disappear from visual cortex because of ongoing visual stimulation (Rademaker, Chunharas, & Serences, Reference Rademaker, Chunharas and Serences2019). Perceptual information is often recoded in the service of specific tasks. Kwak and Curtis (Reference Kwak and Curtis2022) showed subject clouds of moving dots and also oriented gratings, asking them to remember the directions. They found that brain decoding on either of these working memory representation worked on the other suggesting that working memory coded what was in common to the two kinds of percepts, eliminating the moving dots and the gratings, replacing them with representations of vectors, showing that many iconic features can be altered or discarded in working memory.
Of course perceptual working memory is constantly interacting with perception. Quilty-Dunn (Reference Quilty-Dunn2023) argues convincingly that this interaction is crucial to longer term perceptions, for example, perceptions that span saccades. As Quilty-Dunn notes, perception does not start anew after each saccade, so there must be some perceptual – I would say iconic – information preserved by the saccade. True, but there is plenty of evidence for at least some loss of iconicity in transsaccadic memory (summarized in Block, Reference Block2023a, pp. 261–262). For example, in the famous Sperling effect, a multirow array of letters is presented briefly, but a cue presented after the stimulus stops can focus attention on any one of the rows, allowing reporting of all or almost all the items. However, if the array is presented before the saccade and the cue presented afterward, the Sperling effect disappears, showing that transsaccadic memory can erase the iconic memory that the Sperling effect depends on.
Quilty-Dunn describes a perceptual effect known as the motion repulsion illusion. If dots moving in one direction are superimposed on dots moving in a different direction, the perceived angle between the two directions is exaggerated in perception. Kang et al. showed that the same effect occurs if one set of moving dots is seen while the other is held in working memory (Kang, Hong, Blake, & Woodman, Reference Kang, Hong, Blake and Woodman2011). This result suggests that there are iconic elements in perceptual working memory but does nothing to show that perception is not iconic.
As Quilty-Dunn notes, perception can distort working memory and conversely. Teng and Kravitz (Reference Teng and Kravitz2019) showed that colors and orientations in each of perception and working memory affect the other, commenting that this is no doubt because of overlapping representations in visual processing areas. However, as the authors note, these results are compatible with the involvement of prefrontal cortex in working memory. The overlap of sensory coding between visual working memory and vision does not preclude partial reformatting in working memory or the inclusion of iconic information in a discursive envelope.
In sum, perceptual working memory can preserve some aspects of iconic perceptual representation even if it includes it in a discursive envelope. Quilty-Dunn et al.'s results depend on the discursive envelope, not the iconic perceptual representations.
Financial support
The author acknowledges Templeton World Charities for support.
Competing interest
None.