Morin stipulates that self-sufficient ideographies that are understandable to others are rare because they are stifled by the issue of standardization and need to be explained through other means. We agree with this point and argue that mathematical indices and artificial intelligence can assist us in decoding ideography, and if not to understand its meaning, at least to know that meaning exists. Issues such as a lack of representativeness (Martinet & Pelé, Reference Martinet and Pelé2021) by an external viewer arise, of course, with abstract art and graphic productions created by young children who are not yet able to speak and explain their art. Moreover, examining the evolution of ideographies, which may be a first language before writing, is difficult because no other species, except for Homo sapiens, exist in the Homo genus. However, it is possible to mathematically study the ontogeny of ideographies, the scribblings, drawings, or sketches of children and its phylogeny through apes that have the ability to draw.
Morin refers to Ted Chiang's novel, Story of Your Life, in which aliens draw spherical stains or semagrams. Linguists and other scientists have analyzed this alien language using what appear to be network analyses and mathematical tools that were developed to understand how words are linked in the human language (Batagelj, Mrvar, & Zaversnik, Reference Batagelj, Mrvar and Zaversnik2002; Cong & Liu, Reference Cong and Liu2014), and how animal sounds are linked in vocal sequences (Allen, Garland, Dunlop, & Noad, Reference Allen, Garland, Dunlop and Noad2019; Weiss, Hultsch, Adam, Scharff, & Kipper, Reference Weiss, Hultsch, Adam, Scharff and Kipper2014). These studies show that the sequences of vocalizations are not random and that some calls are similar to logical connectors in human language because they are central to the vocalizations network. The linguistic approach adopted in human communication can be used to better understand child or animal vocalizations, writings, or drawings. Fractals in mathematics can be used to comprehend the complexity of physics, economy, and urbanism, as well as the complexity of animal behavior in terms of temporal sequences and spatial distribution as with Lévy flights (MacIntosh, Reference MacIntosh2014). We applied these indices for the first time to drawings created on touchscreens by chimpanzees, children, and adult humans (Beltzung et al., Reference Beltzung, Martinet, MacIntosh, Meyer, Hosselet, Pele and Sueur2022a; Martinet et al., Reference Martinet, Sueur, Hirata, Hosselet, Matsuzawa and Pelé2021). We analyzed the trajectories of finger drawings as animals moving in nature, foraging, resting, and so on. We demonstrated that chimpanzees do not draw randomly but have a lower efficiency than 3-year-old human children. The efficiency of drawings (measured with a spatial fractal index based on trajectories) increases from 3-year-old to 10-year-old children before reaching a plateau during adulthood because adults add too many details to their graphical productions. Regarding ideography, this result suggests that the drawings that are understandable at a glance are those with few details as sketches and that there may be a kind of selection of codes. The temporal complexity of drawings (based on sequences of drawings and nondrawings) follows a similar pattern.
Combining these fractal indices with other measures, such as the entropy or Gini index, we differentiated the scribbles of children from those of adults; it was not possible to discern these differences with the human eye (Sueur, Martinet, Beltzung, & Pelé, Reference Sueur, Martinet, Beltzung and Pelé2022b). Three interpretable dimensions were highlighted in the drawings: Efficiency, diversity, and sequentiality. This means that by observing the coordinates of graphical communication, a drawing, sketch, or semagram, on a three-dimensional graph, we could precisely determine whether the production has meaning and potentially link it to the age, or even the country, of the human creator. Efficiency can be defined as an ability to avoid wasting materials, energy, efforts, and so on in an activity. In the context of graphical communication, we suggest that efficiency is representativeness combined with few details (i.e., optimality). Diversity is almost exclusively linked to the use of colors, whereas sequentiality is linked to graphical complexity. When the complexity of the drawing is increased to make something representative, the number of sequences and the stochasticity increase. Of course there is an interpretation bias in these studies linked to anthropomorphism and the fact that the graphical productions were created by hand (Sueur, Beltzung, & Pelé, Reference Sueur, Beltzung and Pelé2022a). Extending this work to orangutans (Pelé et al., Reference Pelé, Thomas, Liénard, Eguchi, Shimada and Sueur2021), we showed that a female orangutan exhibited a higher graphical complexity than her congeners, and also changed the colors and shapes of drawings (circle, triangles, or fan patterns) according to the seasons/context, which indicated that there was meaning in her drawings. For instance, the subject used red extensively after the birth of an orangutan baby and blue and yellow after the visit of schoolchildren who were wearing yellow hats and blue coats. This study's findings were verified using deep-learning models (Beltzung, Pelé, Renoult, Shimada, & Sueur, Reference Beltzung, Pelé, Renoult, Shimada and Sueur2022b). The convolutional neural networks used in orangutan drawings showed a seasonal effect on the style and content of the productions.
These results indicate that the puzzle of ideography can be extended to the sketches or drawings of children and great apes. The premises of semagrams can already encode meaning without requiring language. Our contribution proposes to go deeper into the ontogeny and phylogeny of communication. Mathematics can decode many nonunderstandable codes for human senses and cognition, and the methodology we applied to understand representativeness in apes' and children's graphical productions can be extended to domains such as comparative psychology, anthropology, and semiotics. We believe that we can create elements and research frameworks for the future of ideography using new technologies, analytics, and support. New technologies combined with new mathematical methods appear to be extremely useful and provide new possibilities to test mental states, intentions, and emotions beyond graphical representations (Watanabe & Kuczaj, Reference Watanabe and Kuczaj2012). Touchscreens score the coordinates of each drawing point, eye-tracking measures anticipation, and encephalograms or magnetic resonance imaging (MRI) can detect whether the communication is indeed a form of language. Finally, our research framework can be extended to psychopathologies such as autism (Jolley, O'Kelly, Barlow, & Jarrold, Reference Jolley, O'Kelly, Barlow and Jarrold2013) or simply be used to measure learning processes and creativity in language or writing (Lee & Hobson, Reference Lee and Hobson2006).
Morin stipulates that self-sufficient ideographies that are understandable to others are rare because they are stifled by the issue of standardization and need to be explained through other means. We agree with this point and argue that mathematical indices and artificial intelligence can assist us in decoding ideography, and if not to understand its meaning, at least to know that meaning exists. Issues such as a lack of representativeness (Martinet & Pelé, Reference Martinet and Pelé2021) by an external viewer arise, of course, with abstract art and graphic productions created by young children who are not yet able to speak and explain their art. Moreover, examining the evolution of ideographies, which may be a first language before writing, is difficult because no other species, except for Homo sapiens, exist in the Homo genus. However, it is possible to mathematically study the ontogeny of ideographies, the scribblings, drawings, or sketches of children and its phylogeny through apes that have the ability to draw.
Morin refers to Ted Chiang's novel, Story of Your Life, in which aliens draw spherical stains or semagrams. Linguists and other scientists have analyzed this alien language using what appear to be network analyses and mathematical tools that were developed to understand how words are linked in the human language (Batagelj, Mrvar, & Zaversnik, Reference Batagelj, Mrvar and Zaversnik2002; Cong & Liu, Reference Cong and Liu2014), and how animal sounds are linked in vocal sequences (Allen, Garland, Dunlop, & Noad, Reference Allen, Garland, Dunlop and Noad2019; Weiss, Hultsch, Adam, Scharff, & Kipper, Reference Weiss, Hultsch, Adam, Scharff and Kipper2014). These studies show that the sequences of vocalizations are not random and that some calls are similar to logical connectors in human language because they are central to the vocalizations network. The linguistic approach adopted in human communication can be used to better understand child or animal vocalizations, writings, or drawings. Fractals in mathematics can be used to comprehend the complexity of physics, economy, and urbanism, as well as the complexity of animal behavior in terms of temporal sequences and spatial distribution as with Lévy flights (MacIntosh, Reference MacIntosh2014). We applied these indices for the first time to drawings created on touchscreens by chimpanzees, children, and adult humans (Beltzung et al., Reference Beltzung, Martinet, MacIntosh, Meyer, Hosselet, Pele and Sueur2022a; Martinet et al., Reference Martinet, Sueur, Hirata, Hosselet, Matsuzawa and Pelé2021). We analyzed the trajectories of finger drawings as animals moving in nature, foraging, resting, and so on. We demonstrated that chimpanzees do not draw randomly but have a lower efficiency than 3-year-old human children. The efficiency of drawings (measured with a spatial fractal index based on trajectories) increases from 3-year-old to 10-year-old children before reaching a plateau during adulthood because adults add too many details to their graphical productions. Regarding ideography, this result suggests that the drawings that are understandable at a glance are those with few details as sketches and that there may be a kind of selection of codes. The temporal complexity of drawings (based on sequences of drawings and nondrawings) follows a similar pattern.
Combining these fractal indices with other measures, such as the entropy or Gini index, we differentiated the scribbles of children from those of adults; it was not possible to discern these differences with the human eye (Sueur, Martinet, Beltzung, & Pelé, Reference Sueur, Martinet, Beltzung and Pelé2022b). Three interpretable dimensions were highlighted in the drawings: Efficiency, diversity, and sequentiality. This means that by observing the coordinates of graphical communication, a drawing, sketch, or semagram, on a three-dimensional graph, we could precisely determine whether the production has meaning and potentially link it to the age, or even the country, of the human creator. Efficiency can be defined as an ability to avoid wasting materials, energy, efforts, and so on in an activity. In the context of graphical communication, we suggest that efficiency is representativeness combined with few details (i.e., optimality). Diversity is almost exclusively linked to the use of colors, whereas sequentiality is linked to graphical complexity. When the complexity of the drawing is increased to make something representative, the number of sequences and the stochasticity increase. Of course there is an interpretation bias in these studies linked to anthropomorphism and the fact that the graphical productions were created by hand (Sueur, Beltzung, & Pelé, Reference Sueur, Beltzung and Pelé2022a). Extending this work to orangutans (Pelé et al., Reference Pelé, Thomas, Liénard, Eguchi, Shimada and Sueur2021), we showed that a female orangutan exhibited a higher graphical complexity than her congeners, and also changed the colors and shapes of drawings (circle, triangles, or fan patterns) according to the seasons/context, which indicated that there was meaning in her drawings. For instance, the subject used red extensively after the birth of an orangutan baby and blue and yellow after the visit of schoolchildren who were wearing yellow hats and blue coats. This study's findings were verified using deep-learning models (Beltzung, Pelé, Renoult, Shimada, & Sueur, Reference Beltzung, Pelé, Renoult, Shimada and Sueur2022b). The convolutional neural networks used in orangutan drawings showed a seasonal effect on the style and content of the productions.
These results indicate that the puzzle of ideography can be extended to the sketches or drawings of children and great apes. The premises of semagrams can already encode meaning without requiring language. Our contribution proposes to go deeper into the ontogeny and phylogeny of communication. Mathematics can decode many nonunderstandable codes for human senses and cognition, and the methodology we applied to understand representativeness in apes' and children's graphical productions can be extended to domains such as comparative psychology, anthropology, and semiotics. We believe that we can create elements and research frameworks for the future of ideography using new technologies, analytics, and support. New technologies combined with new mathematical methods appear to be extremely useful and provide new possibilities to test mental states, intentions, and emotions beyond graphical representations (Watanabe & Kuczaj, Reference Watanabe and Kuczaj2012). Touchscreens score the coordinates of each drawing point, eye-tracking measures anticipation, and encephalograms or magnetic resonance imaging (MRI) can detect whether the communication is indeed a form of language. Finally, our research framework can be extended to psychopathologies such as autism (Jolley, O'Kelly, Barlow, & Jarrold, Reference Jolley, O'Kelly, Barlow and Jarrold2013) or simply be used to measure learning processes and creativity in language or writing (Lee & Hobson, Reference Lee and Hobson2006).
Financial support
This work was supported by an interdisciplinary grant from the CNRS (French National Scientific Research Center, MITI) and an IDEX exploratory research grant from the University of Strasbourg.
Competing interest
None.