Prologue

As mentioned in the Preface, it is now a well-known, but still remarkable, fact that in 2015 the Oxford Dictionary’s Word of the Year was an emoji – the “face with tears emoji” (Figure 1.1). The dictionary’s rationale for this selection was that the emoji was one of the most used new “words” at the time and thus meritorious of its word-of-the-year status.

Figure 1.1 Face with tears emoji

But what kind of “word” is an emoji? It certainly is not the same kind that an alphabetic word is, which is a representation of the sounds of a spoken word – that is, it stands for its phonemic structure. On the other hand, the term emoji, Japanese for “picture word,” is a visual sign standing for a referent or concept directly, with no intervening phonemic (or other kind of word-based) representation. Emoji are used typically (if not exclusively) in informal digital texts together with written words – a hybrid mode of writing where both types of words (alphabetic and emoji) are used in a systematically complementary way, semantically and discursively (Reference DanesiDanesi 2016; Reference SeargeantSeargeant 2019).

Since the Oxford Dictionary event, Unicode (an international encoding standard for different scripts) has made a large repertoire of emoji signs available for installation on mobile device keyboards, on apps, and so on, thus allowing for a routinization of hybrid writing and, in some contexts, even the exclusive use of emoji, such as the translation of entire novels into emoji characters. Emoji are visual signs, as mentioned, but they should not be equated with pictographic writing forms, developed over long periods of time, and used in various countries. Emoji are artificially created (or digitally produced) forms that are intended to be language-independent – that is, a user of any language background can select the emoji from preestablished lists to insert within a digital text. As a result, writing now involves two previously separate competencies or literacies – verbal (language-dependent) and visual (generally less dependent on a specific culture, as in art). This has had a broad range of implications for our conceptions of what communication and literacy are, as will be discussed subsequently (Reference AlshenqeetiAlshenqeeti 2016; Reference DavisDavis 2021; Reference Logi and ZappavignaLogi and Zappavigna 2021). Hybrid literacy also has had implications in various domains of society that have traditionally been based on print literacies, including education, which has typically been seen as the channel for imparting and preserving print forms of literacy. Clearly, the advent of emoji into everyday digital discourses has raised some rather profound questions, some of which will be broached initially in this chapter, and examined in more detail in subsequent chapters. The main aim here is to assess the emoji phenomenon generally, including how it has evolved, and what kinds of research findings it has generated. The studies have shown, overall, that emoji signs not only add semantic nuances to messages but also provide a mode of conveying ideas that retrieves visuality as a form of mind. The research has also shown that emoji are not interpreted and used in the same way, as was anticipated by Unicode; rather, like any human semiotic system they involve polysemy, ambiguity, and other variable meaning modalities.

It has become obvious that – at least for the present – emoji can no longer be considered an ancillary set of picture words for sprucing up informal written messages visually. They constitute a veritable discourse and semiotic code, introduced into everyday (informal) communications by digital technologies. As such, they have led to a need to revisit notions such as literacy and communicative competence, as well as what creativity entails, given that using the hybrid code involves an imaginative sense of how to marry verbality and visuality in a blended way, so as to create a unitary account of some message.

Background

As intended by their creators, emoji have spread globally. But they did not arise in a historical vacuum. They would hardly have become part of writing practices in alphabet-using cultures in the first place without various visual-representational trends that took shape in the late nineteenth and early twentieth centuries. One of these was the advent and spread of the comic book, starting in the late 1930s, which blends images and words into a narrative framework. Artistic movements such as Dada and Futurism, among others, also emerged in the first half of the twentieth century, showing that the printed (phonetic) word, laid out in a linear fashion, was losing its supremacy and uniqueness as a mode of representing and communicating thoughts. These conditioned people to become gradually habituated to visual forms as part of representational activities, loosening the grip that print literacy had on society since at least the 1500s.

Emoji writing has retrieved the tendency to represent the world visually, an anthropological fact that is evidenced by the visual images and figures carved or painted on the roofs and walls of ancient caves as well as on artifacts such as vases. These were our earliest picture writings. From the capacity to represent the world visually, ancient writing systems emerged in the first civilizations of the world. These developed three main types of picture signs – pictograms, ideograms, and logograms – before the advent of alphabets. The pictogram stands directly for its referent through some form of resemblance, simulation, or imitation. The degree of resemblance is higher for concrete referents than it is for abstract referents. Abstract referents came to be represented more specifically by ideograms. These also bear resemblance to their referents, but assume much more of a conventional knowledge of the relation between picture and concept on the part of the user. International symbols for such things as public telephones and washrooms today are modern-day descendants of ideographic representation. Over time, as increasingly abstract signs were required, new forms, such as logograms, were developed. These combined elements of pictography and ideography. For example, the Chinese logogram for east is a combination of the pictograms for sun and tree.

Now, in their own artificial way, emoji retrieve the same range of referential modalities of ancient writing systems, albeit highlighting modernized, comic-book (and often humorous) ways of representing concepts, which is what makes them functionally different from the evolutionary forms of picture writing (Table 1.1).

Table 1.1 Emoji forms and their uses

Pictograms	These stand directly for their referents by some form of resemblance: simulates in outline form the happy smiling face; represents a (modern-day) house via a schematic resemblance to its structure.
Ideograms	These stand for abstract referents via various degrees of symbolism: alludes to snoring (showing the presumed sounds made in snoring); indicates “being in a hurry,” recalling comic-book balloons that represent some sound or action.
Logograms	These combine pictographic and ideographic elements to produce phraseological forms: (I) is a homophonic emoji (eye = I phonetically); represents a heart (pictographically), which is also used commonly as a symbol for love (ideographically). These require conventional knowledge of the relation between the emoji and its referent. And it is mainly in this domain of reference that emoji produce most of their misinterpretations (to be discussed subsequently).

The scripts of early civilizations eventually developed phonographic elements within them – forms standing for parts of words, such as syllables or individual sounds. A complete phonographic system for representing sounds is the alphabetic one. The first such system emerged in the Middle East and was transported by the Phoenicians to Greece around 1000 BCE. It contained signs for consonant sounds only. In Greece, signs for vowels were added to it, making the Greek system the first full-fledged alphabetic one. The transition from visual to alphabetic writing was not revolutionary; it was evolutionary, developing spontaneously in the marketplaces of the ancient world because it made the writing of transactions rapid and efficient. Every alphabet character is the residue of a stylistic alteration to some earlier visual sign (such as a pictogram). Consider the alphabet character “A” as a case in point. It started out as a pictogram of the head of an ox in Egypt (and other parts of the Middle East). The ox pictogram came, at some point, to be drawn only in its bare outline. It was this outline that came eventually to stand for the word for ox (aleph). Around 1000 BCE Phoenician scribes, who wrote from right to left, drew the ox outline sideways (probably because it was quicker for them to do so). The slanted Phoenician figure stood just for the first sound in the word (aleph), because it became very familiar. The Greeks, who wrote from left to right, turned the Phoenician figure around the other way. Around 500 BCE, as such “abbreviated picture writing” became more standardized and letters stopped changing directions, the “A” assumed the upright position it has today, derived from Roman script.

Archeologist Reference Schmandt-BesseratSchmandt-Besserat (1978, Reference Schmandt-Besserat1992) found that the first commonly used pictographic forms were probably made by clay tokens, which were image-reproducing molds. It is not a stretch to say that the emoji keys on keyboards, websites, and apps are contemporary counterparts of the clay tokens. Actually, the emoji sign system is an endpoint in the evolution of emoticons and kaomoji – respectively, representations of a facial expression such as :-) (smile), formed by various combinations of keyboard characters, and emoticons that use a combination of Japanese characters, Roman alphabet letters, and punctuation marks to represent a facial expression vertically (Reference EvansEvans 2017): (n_n), (^_^), and *(^o^)*, for example.

The psychological difference between alphabetic and emoji script is, as mentioned, a difference between the psychology and semiotics of verbality and visuality, between processing words as discretely composed signs (phonemically) and holistically created signs. The importance of visuality to human knowledge-making and communication was discussed at length by Reference ArnheimRudolf Arnheim (1969), who challenged the traditional differentiation between “thinking” (associated with language) and “perceiving” (associated with visual art), claiming that the two have been artificially separated, since they have always co-occurred in human affairs. For Arnheim, visual forms such as paintings are what allow us to have a true understanding of our experiences of the world. In this psychological explanatory framework, it can be argued that emoji have spread so broadly because they have retrieved visuality in everyday communication, used in tandem with other representational structures.

Visuality is assumed to be a more understandable mode than verbality in crosslinguistic and cross-cultural communications, with supposed minimal degrees of interpretive variance. As research by Unicode has shown, facial emoji, designed to resemble facial expressions in suggestive outline form, are the most commonly used ones across the world because of their understandable iconic modalities (that is, their resemblance to facial expressions). But their meanings are not static. For instance, the happy face smiley can be used beyond its initial meaning of happiness to communicate differential connotations and semantic nuances in the context of the intent of a particular message, such as conveying irony. In some countries, such as China, the same emoji might convey mockery instead, given that the eyes on the emoji do not move, as they do on a real face; and this might be interpreted as suppressing a smile, not manifesting it. In the same culture, it could also mean that an interlocutor no longer wants to continue an interaction (Reference Jaeger and AresJaeger and Ares 2017). So, while the smiley emoji is assumed to be conducive to a straightforward interpretation, no matter what native language interlocutors speak, it reveals an inherent law of semiotics – diversity in interpretation is a product of using all sign forms. Nevertheless, this emoji (and most other facial emoji) has had a broad acceptance across the world, indicating that its intended meaning becomes accepted broadly, by force of constant intercultural usage, reflecting a kind of semiotic law of normalization via pragmatic utilization.

Specific design strategies to make the emoji code as universally understandable and culturally neutral as possible were adopted initially. For example, the use of yellow to color most of the facial emoji and the deployment of nondistinct facial signifiers (standing for the mouth, the eyes, facial shape, and the eyebrows) were stylistic attempts intended, arguably, to remove recognizable facial features associated with human diversity. But, almost right after the spread of the emoji code into common usage, Unicode was criticized for not reflecting the diversity of its users. So, new facial emoji, including color options, were designed on purpose to reflect culturally based, ethnically sensitive, and gender-flexible referents. Interpretive variance is manifest especially among the ideographic and logographic emoji; but it can also occur in pictographic forms. For example, the emoji of an airplane could be interpreted as referring to the plane itself, flying, traveling, or being at an airport. One cannot assume an isomorphic relation between the emoji and its anticipated interpretation, even by speakers of the same language, and even within the constraints imposed on interpretation by context. Differential interpretations of the same emoji might also be shaped by the platform used – two people looking at a facial emoji on different platforms might interpret it differently.

A now famous example of misinterpretation due to platform-based variance occurred in 2020, when British actress Jameela Jamil posted a tweet using the face-with-hand-over-mouth emoji as a social commentary on food-challenged people shopping during the coronavirus pandemic (Reference RathoreRathore 2022). On Apple’s iOS, the emoji conveys a pensive (worried) feeling state, but on other platforms it appeared as a giggling face, leaving many receivers of the tweet upset, believing that Jamil was mocking poor people, even though this was not her intended meaning.

The ways in which the emoji code generates meaning have been examined across disciplines since 2015. The gist of the work indicates that they provide a grid of visual-holistic meanings that guides the interpretation of a hybrid message, constituting an annotative template to its meaning. Some work suggests that they project the “body language” that accompanies face-to-face interaction onto the screen. While this is not the only function of emoji, it does seem to have plausibility in some uses. In this area, David McNeill’s notion of gesticulants (1992, 2005), which he adopted from Reference Kendon, Seigman and ElmsfordAdam Kendon (1972), is a useful one. These are spontaneous co-speech gestures that depict the content of utterances, bringing out images that cannot be shown overtly in verbal speech, as well as the mental images of what the speaker is thinking about.

McNeill subdivided gesticulants into five main categories, against which various emoji can be mapped:

1. (1) Iconic: These mirror or simulate in gestural form what is being said. Common examples are the circular hand movements when talking about roundabout reasoning, moving the hands far apart when talking of something large, moving both the head and hands in an upward direction when saying “Let’s go up,” and so on. Emoji standing for iconic gesticulants are common:

1. (2) Metaphoric: These are accompanying gestures that depict the image schemas underlying metaphorical utterances. Examples are the kinds of gesticulants that are used spontaneously with conduit metaphors, with the hands put forward as if they are passing on something to someone or receiving something from someone – “presenting an idea,” “putting forth an argument,” “offering advice,” “give it to me,” “this is for me,” and so on. Again, note that the emoji is not always isomorphic to the physical gesticulant but portrays the concept behind it. Below are a few examples:

“That is the point!”	“You got it!”
“You are the problem”	“Hold up a sec”

1. (3) Beat: This is a gesticulant involving hand actions that resemble the beating of musical tempo, whereby speakers flick a hand or fingers up and down, or back and forth, to accompany the rhythmic pulsation of speech. Beats mark the introduction of new concepts in an utterance. They are used commonly when “making points” – “first … second … and lastly”; or in phrases such as “let me repeat this,” which is rendered visually by the emoji below; note that some linguists suggest that any repeating emoji has a beat function, which is plausible in certain contexts, but can be ambiguous in this kind of repetitive form. The emoji below are simple examples of beat gesticulants:

1. (4) Cohesive: Cohesive gesticulants involve using the hands to emphasize that the separate parts of an utterance hold together. They also are used to signal that an utterance is coming to an end or a conclusion, or else that the interlocutor should listen to the final words, as the following emoji imply:

1. (5) Deictic: Hand actions that indicate something that had been mentioned earlier in a conversation, such as waving a hand near the ear and shoulder to indicate that something has passed, are called deictic. These can also be indexes meant to point out (emphasize) something conceptually, as the following common emoji (which also have sign functions, used on walls or devices) indicate:

McNeill’s gesticulant categories are subtypes of illustrators (Reference EfronEfron 1941; Reference EkmanEkman 2003). They can be defined generically as instinctive co-speech gestural movements that illustrate the content (literal and metaphorical) of vocal utterances. As such, they provide a basis for studying how gesture and language form a single integrated system in verbal discourse. However, gesticulant emoji are actually not like the physical ones that instinctively accompany speech, as emoji are selected intentionally to convey a concept, not instinctively as in hand gesturing. As such, their gesticulant functions are sporadic, rather than systematic.

Because emoji are used across languages and by individuals of all backgrounds, with varying degrees of traditional literacy, they present themselves as highly useful and inclusive teaching devices, since they involve no assumptions as to literacy background. This aspect has led to a research study on the part of one of the authors of this book (Reference PetcoffPetcoff 2023) whereby emoji were used as tokens to impart literacy to learners with developmental challenges. The findings and implications will be discussed in the next chapter. Suffice it to say here that using emoji with other digital signs constitutes an out-of-classroom literacy that can easily be brought into the classroom as a pedagogical metalanguage through which literacy training can unfold. It is an example of using the known to reach the unknown via a form of inductive transference, whereby developmental reading and writing students can unconsciously reflect upon how language can be used for academic expressive reasons with the emoji code (Reference GeeGee 1989; Reference StreetStreet 2003; Reference Lea and StreetLea and Street 2006). The Petcoff study showed that emoji literacy can be incorporated pedagogically and andragogically to help students achieve academic literacy, because emoji writing taps directly into the communicative habits of the students, thus operationalizing it as a practical language of instruction in view of laying a path toward a broader form of literacy, as required by schools and society.

Discourse and Semiotic Functions

It is useful to divide the main, or meta, functions of emoji into discourse and semiotic ones – the former refers, simply, to the communicative aspects of emoji use and the latter to the meaning structures and interpretive range of emoji; but in most uses the two overlap considerably. A primary function is the phatic one (Reference Malinowski, Ogden and RichardsMalinowski 1923; Reference Jakobson and SebeokJakobson 1960) – speech used for ritualistic interaction and for keeping social bonds intact or lubricated. For example, a happy face smiley used at the beginning of a text message has the same intent of verbal salutations such as “Hello,” “Dear so-and-so,” and the like. But it does much more than this – it provides an opening interpretive frame for imbuing the tone of the message with positivity, thus ensuring that a phatic bond between interlocutors is established. On the other hand, in a conflictual message, the same emoji would indicate an ironic or sarcastic intent, providing a different interpretive frame for grasping the function of the entire message. This shows how the discourse and semiotic functions can overlap, depending on intent, which is what makes emoji semiotically and discursively powerful.

Another primary function is the emotive one. This is the portrayal (consciously or unconsciously) of the emoji user’s state of mind, mood, attitude, and so on; the desired effect on the receiver that the emoji is designed to have is called the conative function. In face-to-face communication, people use interjections, intonation, and other prosodic strategies, alongside specific words and phrases, to convey their feelings, explicitly or implicitly, and as strategies to elicit reactions in interlocutors. In hybrid digital messages, these functions are rendered by sentiment-based emoji such as faces, hearts, and the like. However, it can be said that the emotive-conative functions are built into virtually every emoji, given that they are chosen not to convey literal information, but to inject emotional tone and semantic nuances into a message. So, a heart emoji placed at the start of a message will tend to imbue the message with “heart-felt” nuances, whether these are authentic or feigned for satirical reasons (Reference Danesi, Petrilli and JiDanesi 2022). The emoji code can thus be said to constitute a “sentiment-placement-based distributive system” – that is, a means to infuse an entire message with sentiment by placing appropriate emotively charged emoji at strategic locations in a message so as to shape the message in terms of intent and emotivity, on both parts. This can be called an “episodic” structure, which is different from verbal syntax or style in the fact that the placement of the emoji signs is not governed by rules of syntax, but rather by unconscious rules of emotivity; that is, emoji are placed in strategic locations among the episodes of a text to inform its meaning in terms of these emoji insertions.

What characterizes an emotive-episodic representational system is that the emotions or moods injected into a message are left unnamed and must be gleaned through the visuality system of understanding (Reference Novak, Smailović, Sluban and MozetičNovak et al. 2015). A key study by Cohn and colleagues (2018) found that emoji placement was not subject to the normal rules of grammar. The researchers examined the type and location of emoji (within clauses, at the start, and at the end), finding that there was a “processing cost” for deciphering images as compared to words – that is, it took more time for subjects to process the meaning of emoji when compared to word placement, determined by syntax, indicating a different mode of processing emoji versus words. However, the kind of implicit knowledge that emoji placement implies indicated that there was no significant difference in comprehensibility between word and congruent-image placements.

Another function of emoji is illustrative, reinforcing the contents of a message via imagery (emoji of a house, a car, etc.); however, even in such uses the emotive-episodic modality built into emoji involves the injection of annotative nuances to their placement. The meaning of “really looking forward to the meeting” changes significantly when accompanied by an eye-rolling emoji, which adds an ironic annotation to the message (Reference RobsonRobson 2019). The cat-face-with-a-wry-smile emoji is an example of how an emoji can be placed episodically in a text to imply irony or mockery visually, no matter what the content may be, evoking the same pattern of understanding that characterizes the decoding of verbal irony (Reference Weissman and TannerWeissman and Tanner 2018; Figure 1.2).

Figure 1.2 Ironic cat face emoji

The choice of a cat (with a wry facial grin) as suggesting irony is consistent with one of the perceived roles of cats in cartoons, comics, and literary traditions. The cat emoji can be traced, arguably, to the figure of the Cheshire Cat – the fictional cat made popular by Lewis Carroll in his children’s novel, Alice’s Adventures in Wonderland (Reference Carroll1865). Carroll’s cat has become known for its distinctive mischievous grin, which is left as a trace after the Cat’s periodic and gradual disappearances. This suggests that an emoji does not sever its connectivity to previous connotative meanings.

Another major function involves the use of emoji as projections of the prosodic-tonal features of face-to-face discourse onto the written text (Reference EvansEvans 2017). In effect, these emoji are body language signs; but they too have episodic function since they can also guide the desired interpretation of a message when placed at certain locations – encoding a mood or sentiment that is distributed into other parts of the message. So, if the intended sentiment is “unhappiness,” then, say, a facial emoji with a frown and closed, downcast eyes, simulating sorrow or pain (), might be used in places where this sentiment is evoked. This sentiment then becomes part of the subtext of the entire message, leaving its emotive traces episodically (Reference Danesi, Petrilli and JiDanesi 2022). Analogously, facial emoji that project humor, pensiveness, contradiction, uncertainty, and the like onto the written message have episodic-distributive functions.

One discourse function that is seemingly missing from emoji use, given that they are employed primarily in informal communications, are the normal sociolinguistic registers – modes of communicating that are designed to match the formality of a situation, the medium used, or the nature of the topic involved in a speech act. In an emoji-using text many of the sociolinguistic categories (register, honorifics, etc.) collapse, or at the very least are not marked as they are in routine communications. This does not mean that they cannot be used to convey politeness. This will be discussed subsequently within the healthcare communications field, where politeness is an expected feature of the interactions.

As mentioned earlier, one of the primary functions of emoji (especially the facial ones) is to reproduce facial microexpressions, according to speech act typologies (Reference SearleSearle 1969, Reference Searle1976), rendering them, however, in humorous-cartoonish ways. A microexpression is a facial expression that lasts briefly, resulting from an emotional response to something. A leading researcher on microexpressions is American psychologist Paul Ekman, a notion he adapted from the key work of Reference Haggard, Isaacs, Gottschalk and AuerbachHaggard and Isaacs (1966) and Reference HjortsjöHjortsjö (1970), adding significant detail to it. Perhaps the reason why facial emoji tend to be the ones most used across the world, and the ones generating less meaning variance, is because they are designed according to universal microexpressions. Ekman and his co-researchers (Reference Ekman and FriesenEkman and Friesen 1971; Reference Ekman, Wagner and MansteadEkman 1989; Reference MatsumotoMatsumoto 1992) found a high level of consistency in various cultures in how they identified the basic microexpressions in virtually identical ways for basic emotions such as embarrassment, anxiety, guilt, pride, relief, contentment, pleasure, shame, and a few others. Illustrative examples of microexpression emoji are shown below:

• “I am pretty sure that this is fishy”:

• “Trust me, this is correct”:

• “This is disgusting”:

• “This is hilarious”:

• “This is amazing!”:

• “I am not amused”:

• “What a relief!”:

• “This is becoming boring”:

• “What an embarrassment”:

The list of such microexpression emoji is endless, and it is being constantly updated and expanded for various speech act functions. The semiotic problem with these (and most other emoji) is that culturally shaped meaning variance crops up from time to time that may lead to misinterpretations in messages. The thumbs-up emoji, , which is still identified as one of the most used emoji across the world, is now a widely known case in point, since the gesture it represents has a whole array of unwanted connotations that users in some non-English speaking countries want to avoid, finding the emoji derogatory or insulting.

Some microexpression emoji are used as gambits – words or phrases used to open a conversation, to keep it going, to make it smooth, to repair any anomaly within it, and so on. The following are common English gambits with corresponding emoji:

• “Uh huh … yeah … hmm … aha … ”:

• This gambit is a filler, allowing an interlocutor to make it known to the other interlocutor that they are in fact in agreement as the message proceeds. The smiley emoji repeated in sequence suggests encouragement to keep an idea being expressed going.

• “I didn’t say that”:

• Negation can be represented in various ways with different kinds of emoji, depending on the intensity desired.

• “You like this, don’t you?”:

• Tag questions ensure consent. The thumbs-up emoji is one way to insert the tag question gambit in a message, albeit with reservations as to the meaning of the thumbs-up gesture (as mentioned).

• “May I ask you something?”:

• Questions designed to gain entry into a conversation can be rendered by various emoji. This one indicates that a thought has come into the interlocutor’s mind (during a repartee) suggesting the desire to interject something into the digital conversation.

• “No way!”:

• Forceful interjections indicate agreement or disagreement patterns during a conversation. The emoji shows how this gambit can be realized in a lighthearted, wry way.

It is to be emphasized that the gambit-focused microexpression emoji selected above reflect subjective choices; indeed, many other emoji could have been chosen to communicate the same type of utterance. This suggests that emoji, like verbal language, vary idiolectally or subjectively (for example, Reference Dainas, Herring, Xie, Yus and HaberlandDainas and Herring 2021).

A salient feature of emoji in general is that they compress meaning efficiently, thus increasing the economy of conversational sequences. One or two emoji can, for instance, be used to answer a question such as: “How are you?” (a happy face emoji, a sad face emoji, a thumbs-up or thumbs-down emoji, etc.). Such one-or-two-emoji responses not only economize on the verbal structural material that would otherwise be required but also minimize the exchange of perfunctory repartees, making communication more efficient and less prone to interactive superfluity. Moreover, the one-or-two-emoji response can fill in gaps of meaning that might otherwise emerge in the course of a communication, often signaled verbally by expressions such as “I am not quite sure how to say this” or “I can’t find the right word.” This implies, more generally, that emoji might be able to combat manifestations of hypocognition, or the inability to communicate certain ideas because there are either no words for them or else the appropriate words cannot be found (Reference LevyLevy 1975).

Interestingly, it has been found that when we look at a smiley face, the same parts of the brain are activated as when we look at a real human face (for example, Reference Churches, Nicholls, Thiessen, Kohler and KeageChurches et al. 2014). While emoji are communicatively efficient when used alone, a study by Scheffler and colleagues (2021) has shed considerable light on how the emotive content of emoji is processed. Subjects were asked to read texts with emoji in them, as the researchers measured the reading time they used to do so precisely. As expected, it took them a little longer to comprehend a sentence that includes emoji than one that does not. This extra time implies that the brain is mapping the emoji against the content of the message via processes that are not unlike those employed in figurative language processing, whereby a concrete referent (for example, sweet) is mapped onto an abstract one (for example, love) to produce a metaphorical thought: love is sweet. The presence of metaphorical meaning in emoji interpretation is a constant one. As Regina Robson remarks (2019): “There is no literal meaning. Emoji are frequently used as a substitution for text, such as inserting multiple hearts in lieu of writing ‘love.’” While comprehending a hybrid message of text and emoji may be a bit more time-consuming, the meaning of the message is more easily and clearly understood during processing, as if speaking to someone in person.

Part of the process of unpacking the meaning of emoji involves deciphering pairing sequences, that is, the interpretation of emoji used in combination with other emoji or specific words and phrases – a feature called adjacency pairing in discourse study (Reference HallidayHalliday 1985). Common adjacency pairings in digital messages include the following:

• The smiley happy face, the heart emoji, and the thumbs-up emoji often appear alongside phrases such as “That’s true,” “I agree”; but they also surface contrastively with statements such as “That’s really not true” and “I wouldn’t say that.”

• The hand-clapping emoji, the hands raised in celebration emoji, and the handshaking emoji are found commonly alongside phrases such as “Yeah,” “Sure,” “Right”; but also contrastively (ironically) with “No-no,” “No way,” “Not true,” and the like.

• The thinking face emoji (with the hand touching the chin), the side eye emoji, and the emoji face with a finger pointing to the head are found frequently alongside disagreement statements such as “Yeah, but, maybe.”

• Tag questions, such as “You agree, don’t you?,” are commonly accompanied by emoji clusters such as the question mark followed by consent emoji.

As Reference Wagner, Marusek and YuWagner et al. (2020) have noted, the hidden power of emoji communication is that it can “fundamentally alter the exchange of emotion [because] images and signs that depict feelings have replaced the nuanced selection of the right words and phrases.” As the authors go on to surmise: “Across the globe, the expressive range of human emotion through the static ideogram of the emoji, or emoticon, presents an increasing challenge for the visually immediate, nonverbal exchange of capricious emotive communication.”

Semioliteracy

The preceding discussion implies several key issues, which will be addressed in this book: How can knowledge of the hybrid writing system be incorporated andragogically (as well as pedagogically) as a way to impart literacy, especially in the case of adult learners? We focus on two very different areas to explore how emoji facilitate communication: learners with developmental needs as well as those involved in healthcare communications. The former explores how emoji are used as a teaching tool; the latter explores the general use of emoji as communicative tools in a specific area of discourse.

The overall principle adopted here is that by using emoji in teaching about literacy, students will not only acquire the ability to grasp the discursive structures involved in literate expression but also be able to use their emoji competence (presentational knowledge) freely to make the path toward gaining literacy a more fluid one. As the Petcoff study has shown, students gain a concrete understanding of the functions of signs (verbal or visual) in their most meaningful (everyday usage) contexts, becoming adept at integrating presentational, discursive, and recursive sign structures to read and write – recursion is used here to mean the repeated application of some structural form or process in different contexts as part of various communicative functions. In textual writing, subject and object pronouns, locative particles, demonstratives, adverbs, and other kinds of so-called deictic forms are recursive structures referencing other parts of speech. Recursive emoji are found constantly in messages, indicating that they have blended distributive and episodic functions (discussed subsequently). Suffice it to say here that they are episodic structures; their insertion in different message locations makes reference to some concept stated before (anaphoric reference) or to some anticipated concept (cataphoric reference). So, if a heart emoji is used before the statement, “I love you,” then it is anaphoric; if it is used after, it is a kind of mnemonic device that functions cataphorically.

Given that emoji involve a blend of discursive and semiotic functions, as already discussed, their utilization falls more precisely under the theoretical framework of semioliteracy (Reference PetcoffPetcoff 2022a, Reference Petcoff2022b, Reference Petcoff2023), a form of literacy based on the ability to integrate signs selected from any modality to create a message – verbal, visual, audio, and so on. Such literacy is now an unconscious one, given the presence of all kinds of sign forms and systems, from emoji to animations to picture memes, that can be used to compose messages. So, the term semioliteracy embraces all kinds of literacies in actu and is thus key to devising pedagogically appropriate literacy instruction tools. As Reference McLuhanMcLuhan (1962) argued, writing practices are based on a specific technology (writing tools) that is shaped evolutionarily by other technologies (such as the printing press). This does not mean shedding previous notions of literacy; on the contrary, it means integrating them with current technologically based literacies. By coordinating traditional literacy with the new literacies, the semioliteracy approach in teaching will allow teachers to tap into ever-evolving student learning styles, so as to balance these with the curricular standards set forth by learning institutions and governing bodies. Within this organic context, the definition of semioliteracy actually matches the definition of literacy put forth by UNESCO as “the ability to identify, understand, interpret, create, communicate and compute, using printed and written materials associated with varying contexts” (Reference MontoyaMontoya 2018).

Concretely, semioliteracy is the ability to comprehend and utilize signs of all kinds (presentational, discursive, recursive) in any technological modality (verbal, visual, audio, animated, etc.) in terms of contextual constraints mapped against traditional forms of print literacy. It encompasses the ability to cultivate an overall semiotic-discursive competence that is required in today’s complex technological world. Within the semioliteracy theoretical framework, it is the hybrid form of writing and reading – a blending of linear phonetic writing with emoji writing – that forms the central approach to instruction. This approach can also be called bimodal, which provides a neuroscientific slant on the pattern of integration that is implied by the combined use of verbality and visuality in writing (Chapter 2). The effectiveness of the semioliterate approach was investigated by Petcoff with a group of developmental reading and writing students at Tarrant County College Southeast Campus in Arlington, Texas. While the Texas Higher Education Coordinating Board defines the general criteria for identifying developmental reading and writing students and schools across Texas, each school was allowed to implement its own relevant instructional programs. One particular model – the Integrated Reading and Writing Model, mandated by the THEC Board – addresses situations where the acquisition and mastery of literacy prove to be more challenging to developmental learners. The semioliteracy framework adopted by Petcoff afforded her an opportunity to circumvent the usual challenges to literacy acquisition that such learners face. But its implications reach beyond, since it can be argued that it is beneficial to students of all backgrounds and needs.

The term semioliteracy actually encapsulates educational-instructional practices that have been developed since the rise of technologically based literacies and thus constitutes a broad rubric for discussing and examining learning outcomes. Already in 2011, Jody Shipka, an English professor at the University of Maryland, introduced a model for operationalizing multimodal literacy as a blend of digital and non-digital modes of expression. As Reference ShipkaShipka (2011) claimed, the distinction between technical (formal) or nontechnical (nonformal) literacy is losing its value in a multimodal world of mass communications. However, she cautions that those who over-enthusiastically adopt digitized texts or computerized teaching tools exclusively as a means of practicing multimodal teaching are shortsighted, given that such training fails to take into account the academic literacy that is still required today. As she argues, multimodality does not involve exclusion of print literacy but rather its amalgamation with the other multimodal literacies reemerging in the world, and is used in tandem with other modes of conveying information. One of her most seemingly bizarre, yet pointedly relevant, examples of her overall approach is her use of pink ballet slipper shoes to teach multimodal composition (Reference ShipkaShipka 2011: 1), as part of what she calls “three dimensional” teaching, whereby the shoes involve meaning exchanges that are relevant to the students, given that they can relate to them semiotically – that is, they are perceived as part of a system of understanding that integrates language, objects, social functions, and so on into a type of instruction that brings elements of the real world into the abstract system of understanding called the classroom. Shipka uses the slippers in the classroom to allow students to envision them as objects (in the semiotic sense) that become a basis of discussion. As (Reference ShipkaShipka 2011: 7) notes, overall:

If we are committed to creating courses that provide students with opportunities to forge new connections, to work in highly flexible ways, and to be increasingly cognizant of the ways texts provide shape for and take shape from the contexts in which they are produced, circulated, valued, and responded to, it is crucial … that we not limit the range of materials or technologies students might take up later in compelling ways.

Shipka’s approach is fundamentally a semioliterate one, and as such it was adopted and adapted by Petcoff, formerly a special education teacher in the Arlington Independent School District and current developmental reading and writing college instructor (Chapter 2). Above all else, she found that this approach reverberated emotionally and intellectually with underprivileged and othered students. The resources to teach them are not as plentiful as those earmarked for the mainstream classroom. So, the developmental reading and writing instructor has always been faced with the need to create self-styled teaching tools from whatever resource is available. Using emoji to teach literacy skills implies providing, at the very least, a repertory of pedagogical techniques and materials for addressing the learning issues that traditional publishers have tended to ignore. As with Shipka’s pink ballet shoes, emoji are effective semiotically because they resonate with the students at different levels of understanding. They offer a means to present abstract concepts via familiar visual signs. Also like the Shipka approach, different media and semiotic sign systems are mapped onto the core emoji-literacy approach, including video, mobile phones, popular culture texts, and so on, so as to create a multimodal learning environment that mirrors and reproduces the communicative environment in which the students find themselves immersed on a daily basis.

Although emoji meanings may vary across cultures and according to operating systems or web platforms, overall, the emoji code comprises a recognizable visual code that is used throughout the world. Traditional literacy instruction tools were crafted on the basis of alphabetic texts, with proficiency and mastery involving the ability to perform basic decoding and encoding processes on such texts. While these skills are certainly needed today, the discursive nature of traditional literacy instruction does not allow learners today to gain access to the structures of print literacy, given that they are constrained in function (to academic texts); and moreover, as media such as YouTube now show, multimodality can no longer be avoided, even in literacy education, since people have become accustomed to seeing, hearing, and reading information traditionally available only in text (Reference SelfeSelfe 2017).

In sum, the semioliteracy framework aims to integrate different sign systems into a coherent holistic model of what understanding texts implies today. It is the core notion in what is called edusemiotics (Reference Semetsky and StablesSemetsky and Stables 2014) – a pedagogical approach in which teaching any subject matter should take into direct account the semiotic notions that undergird sign use. As a simple, yet nontrivial example, teaching math literacy in an edusemiotic way would involve presenting: (1) the signs necessary to carry out some task or to conceptualize some idea, (2) using some visual-semiotic strategy to make ideas more comprehensible, and (3) comparing mathematical signs with other kinds of signs so as to bring out common principles of representation. This is not new, of course, but it was never named as a specific kind of approach. So, the usual diagrams used in mathematics can be seen to be edusemiotic tools in that they present information concretely, and thus much more comprehensibly, by highlighting or showing the essential features of a problem. As Richard Reference SkempSkemp (1971: 101) has put it, a diagram “conveys all that verbal statements do, more clearly and vividly.” In a sense, emoji are the equivalent of mathematical diagrams, since they present information clearly and vividly.

Other components of a semioliteracy-based pedagogy involve reader-transactional theory, media/digital literacy, multimodal composition, and visual literacy. Transactional theory, developed by Louise Rosenblatt (for example, Reference RosenblattRosenblatt 2005), proposes that teaching a learner to read involves both the reader and the text itself in the conceptualization of its meaning – not their separation into distinctive components. The meaning extracted from a text is, therefore, produced by a continuous transaction between the reader and the text, involving above all else the experiential background a reader brings to the text. This shift of focus onto the reader has been a trend within semiotics proper. The late Umberto Reference EcoEco (1979) was among the first to indicate that there is a relation between how a text is interpreted in any era and the nature of the interpreters themselves. Successful interpretation of the text on the part of a reader is not a straightforward matter of trying to determine the text-maker’s intentions, but invariably involves what is in the reader’s mind at the outset. In effect, there is a negotiation of meaning between text-maker and reader that ends up in varying interpretations across social systems and across time. Given their episodic nature, emoji can be used as particular kinds of interpretive frames in text analysis, since they allow for annotation and commentary on the print text via their visual nature, again, much like the illustrations in picture books. This view of texts has shattered the traditional critical approach to interpretation assigned to those with specialized literacy (such as critics), opening textual analysis up to anyone, anywhere (Reference FreireFreire 1970; Reference BourdieuBourdieu 1972). The semioliteracy pedagogical model can be summarized as seen in Figure 1.3.

Figure 1.3 Semioliteracy

This model is intended to take into account diversity of learning styles, background variables, and various other aspects that aim to make textual analysis truly open to everyone. One of the first to realize that semioliteracy was critical in activating learning was the semiotician-psychologist Lev Reference VygotskyVygotsky (1961), who put forth the idea of “zones of proximal development,” implying that learners progress through zones of learning that become extended spontaneously as soon as they are able to understand new input by themselves. He proposed developmental stages that go from external (physical and social) actions toward internal cognitive representations that find expression in the diverse semiotic activities (verbal and nonverbal). As such, he saw a coordination of different literacies as key to the acquisition of knowledge and skill. The student is thus a “creator of knowledge,” rather than a passive consumer. As Reference ShaferShafer (1998: 19) characterizes such approaches, pedagogical optimization is realized when students are allowed to learn “through a gradual and constructive approach.” In the area of literacy instruction, the central idea is to show that reading and writing are not just academic skills, but social and technologically shaped ones as well. While emoji themselves are not marked for register (discussed subsequently), they nonetheless can be used as an access route to the higher registers implied by possessing print literacy.

The movement known as “new literacies studies” took shape in the 1980s as one of the first to put forth a socially based semioliteracy approach at the threshold of the digital age (for example, Reference GeeGee 2015). In contrast to traditional literacy pedagogy, which envisioned reading and writing as an abstract (idealistic) process inside the mind, the members of the movement argued that literacy was something we did in society, not just inside our heads, and should be taught and studied as such. It viewed literacy as a sociocultural rather than just a mental phenomenon. A supporter of this view, Brian Reference StreetStreet (2013), put forth a derived model that he calls, “autonomous literacy instruction,” in which he suggests that literacy outside of any social context is meaningless because it is abstract and self-serving and, thus, is nonfunctional. The main problem with traditional literacy instruction is that the technical functions of literacy are presented as being independent of any concrete social uses. The rote memorization of rules and patterns in reading, writing, and discursive practices is encouraged in a decontextualized fashion, rendering literacy as largely pointless. While Street sees the role of technical knowledge (grammar, phonics, style, etc.) as important, he stresses that it should be taught from a context-dependency point of view, so that the teacher can illustrate the importance of “the broader cultural perception of ways of thinking about and doing reading and writing in cultural contexts at work” (Reference StreetStreet 2003).

In line with the traditional sociosemiotic view of language (including Vygotsky and Halliday), linguist James Paul Reference GeeGee (2015) sees literacy instruction as needing to be connected with contemporaneous social needs and communicative patterns. In other words, literacy makes sense if it is mapped against actual communicative practices. It is in this framework that a semioliteracy approach that incorporates emoji as a nontraditional literacy would fall. Since we live in a world where emoji usage, alongside other digital signs, has become a kind of default mode of writing in everyday communications, the question of whether they can be deployed for educational purposes is clearly a key one. As such, it has led to various pedagogical studies, most of which present positive findings in this regard. Increasingly, the pedagogical utilization of emoji is showing up in different school subjects, from mathematics (Reference Holton and SymonsHolton and Symons 2021) to English (Reference KingKing 2016). Most of the studies indicate that emoji influence the “student’s impressions of credibility, immediacy, and liking,” as Reference Vareberg, Vogt and BerndtVareberg et al. (2022) aptly observe.

To reiterate, the main idea of adopting a semioliteracy framework is to impart academic literacy through the discourse and semiotic functions of emoji, thus making the study of literacy relevant to students who do not learn well within the traditional curricular parameters (Reference Glass and KangGlass and Kang 2018). It also allows students to utilize the devices they bring to class in a constructive way. Studies such as the one by Reference Wood, Zivcakova and GentileWood et al. (2012) have shown, in fact, that excessive and distracting in-class cell phone usage is a problem across various institutions and learning environments (Reference Mapes, Hea, Alexander and RhodesMapes and Hea 2018). Giving verbal instructions as to how to complete an assignment or answer questions about the instructions tend to be ignored, because of the availability of mobile phone communications. This has often led to the need for remedial pedagogy – which itself has hardly ever turned out to be effective. So, the task set out by the Petcoff project was to envision how to incorporate mobile devices directly into the classroom, thus not going against the grain but rather inserting oneself pedagogically within it. The task was, however, complicated by the fact that the school system required procuring approved teaching tools; that is, tools that met the needs of both the individual (and challenged) student while being accessible and relatable to the adult learner demographic. The project found that the connecting link between the two sets of learners was emoji communication. It showed that the semioliteracy model was the most effective one because it allowed for the integrated use of discursive, presentational, and recursive aspects of literacy training. The approach is one potential response to the Conference on College Composition and Communication’s (CCCC) 1974 Students’ Right to Their Own Language declaration, which was intended to enact a strategy of inclusion of all students, including struggling ones.

The CCCC is a national professional association of college and university writing instructors in the United States. It was established in 1949 as an organization within the National Council of Teachers of English and continues to be one of the largest organizations dedicated to literacy research, theory, and teaching worldwide. A key aspect of CCCC-based research is to seek out teaching tools that are geared toward a diverse population of students, while avoiding instruction that would be perceived as demeaning or even just subpar with respect to the general student population. Given the common use of emoji as visual communicative signs, complementing verbal ones, the claim made here is that they can bridge the gap between student populations effectively, given their relevance to all types of learners (especially outside the classroom). Petcoff’s goal was to bring emoji into the developmental (remedial) education classroom so that pedagogy could tap into how students used them to communicate, think, and create meaning on their own, outside of traditional literacy-based rules and conventions.

The term developmental education was used for the first time in 1970 by Frank Christ to describe the kinds of pedagogical efforts developed to teach students in his Learning Assistance Center (Reference Arendale, Boylan and BonhamArrendale 2014: 15–16). His idea was to assist students with learning difficulties, with special needs, or from underprivileged backgrounds to achieve expected competencies in core academic skills such as literacy and numeracy via pedagogy designed specifically for them. However, by the millennium, the typology of developmental students had become more complex, as the student population became more and more multiethnic and multiracial, differentiating itself from the primarily white population of the mid-1940s to the early 1970s, when, as Reference Arendale, Boylan and BonhamArrendale (2014: 19) notes, the developmental categories were enlarged to encompass “traditional white male students, nontraditional males and females, and federal legislative priority groups: first-generation college, economically-disadvantaged, and students of color,” as well as older students returning to education or attending college for the first time. Today, this diversity of is no longer an exception, but a reality in all kinds of classroom situations. More recent scholarship has further expanded the typology (Reference Hassel and PhillipsHassel and Phillips 2022: 11), embracing any group of learners with the common need to improve the basic literacy skills that have been mastered by nondevelopmental peers.

Like the new literacy approach, semioliteracy pedagogy envisions all types of students as equal learners. In the current technological environment, notions of literacy are changing and adapting in response to the ways people communicate globally. A primary consequence of this reality is the need for what is now called “situated practice,” which focuses on the connection between classroom learning and real-world experiences, building upon the concrete personal experiences of learners. Another consequence is the need for the critical framing of knowledge, whereby the sociocultural contexts in which a concept (in history, literature, science, mathematics, etc.) is developed is given prominence via semioliterate techniques. Finally, the crux for updating pedagogy is to adopt all kinds of sign systems as part of a semioliteracy approach. It is in turning theory into practice that emoji-based instruction comes concretely into play, since it allows for the student’s emoji literacy to become an intrinsic part of learning other kinds of literacies. Emoji bring learner experiences, knowledge, and interests to the artificial classroom learning environment, allowing them to include their perspectives directly into the learning process. This activates Vygotsky’s zone of proximal development, since the learners are immersed in taking part in something that is new or unfamiliar, but within the zone of their own life-worlds.

Semioliteracy teaching falls into the tradition called “authentic pedagogy,” formulated in counter-position to “didactic pedagogy,” traced back to the early twentieth century ideas of John Dewey in the United States and Maria Montessori in Italy (Reference Kalantzis and CopeKalantzis and Cope 2012: 95–117). The central notion is a focus on the learner’s own experiences and talents, adopting materials that are relevant to their everyday lives. When it comes to reading and writing, authentic literacy pedagogy promotes internalized understanding rather than the abstract formalities of grammar and vocabulary. In the context of our increasingly diverse and globally interconnected lives, authentic literacy cannot be assigned only to developmental or specialized learning, since everyone is immersed in the same communicative reality. The traditional curricula operate on various rules of inclusion and exclusion, which involve a hierarchical ordering of literacy practices, often dismissing textual types such as picture books or popular fiction. In so doing, however, groups of learners are disenfranchised and may lose out on opportunities to achieve literacy practically.

In sum, semioliteracy transcends conventional print literacy pedagogy via the adoption of the different modes of communicating today. The use of emoji, which are part of a quasi-universal mode of communicating, allows for a form of situated learning that can potentially lead to the mastery of such literacies (as will be discussed). It moves away from the assembly line–based pedagogy of the past to a more personalized one that would unfold within the Vygotskian zones of proximal development, allowing the learners to progress at their own pace but within contextualized limits.

Applications to Healthcare Education

Semioliteracy is a notion that is best related to a “semiotic consciousness,” whereby the different modes and media of sign use come into play in all forms of communication and understanding. It is also the basis for traditions and dialogical practices in medicine, given the connection among health, disease, and sign systems. In an age when specialized medical approaches to disease and health can no longer be isolated from the technological cultures and situations in which they occur, a goal of emoji-based practices and professional training in the public healthcare system is to enhance the ways of conveying to the general public the meaning of diseases through a visual communicative template. The Palaganas studies aiming to assess this approach are discussed in detail in Chapter 4. Suffice it to say here that this has practical implications for healthcare teams and the conduct of public health practices. The ancient Greek physician Hippocrates – the assumed founder of both medicine and semiotics – put it insightfully as follows: “It’s far more important to know what person the disease has than what disease the person has” (cited in Reference AdamsAdams 1849: 23).

Given their high emotivity, as discussed, one area in which emoji use is receiving particular attention is in physician–patient interactions. Reference Renne, He and LeeRenne et al. (2022) claim that it allows physicians and patients to better understand each other, making symptoms and clinically relevant information visible (literally). Dr. Shuhan He was actually a cocreator of the anatomical heart and the lung emoji introduced globally in 2020. He was also one of our interviewees for this book (as mentioned). The capacity of emoji visuality to convey meaning powerfully emerged during the coronavirus pandemic, when cartoonish emoji-like pictographs of the virus became Internet memes. An example can be seen in Figure 1.4.

Figure 1.4 Coronavirus emoji

The above (made-up) emoji (similar to those on the Internet) transmits in a comical way what the health enemy looked like. Aware of the emotive power of such images, Graphic Medicine, a health platform, adopted visuality systematically as a means to communicate with the general public, devising emoji, comics, and cartoons for informing or warning people about the effects of a disease (Reference TuohyTuohy 2018). Interestingly, the US-based Centers for Disease Control (CDC) hired medical illustrators to create a visible identity for the coronavirus that would draw attention to its danger via the power of iconic presence, perhaps inspired by the emoji used in this area of digital communications. Whether the emoji was actually effective or not during the height of the pandemic to get people to adopt prophylactic measures is a moot point – it became, as Reference SonnevendSonnevend (2020: 461) remarks, “a key [source] for imagining the crisis.”

Emoji rather than text are useful in public health messaging, especially when communicating in situations where multiple languages are involved. Moreover, they might be more effective stimuli for getting people to adopt healthy practices than admonitions via words. A study by Reference Lotfinejad, Assadi, Aelami and PittetLotfinejad et al. (2020) found that emoji were useful in global health efforts to get people to wash their hands: “Concerning the infiltration of emojis into scientific literature, it is imperative to evaluate the impact of these symbols in health-related fields in order to harness their potential advantages for appropriate research applications and stave off scientific miscommunication.” In situations other than public health messaging, what keeps doctors from using emoji with patients is that there is not a clear context or set of rules of etiquette for them – to be discussed in Chapters 3 and 5.

Epilogue

The fact that emoji have spread across the world is evidence that alphabet-based writing is no longer the exclusive mode. Reference McLuhanMcLuhan (1964) had observed long before the Internet age that the capacity for daily global communications would create a sense of being caught permanently in the present, implying that there is an immediacy to such communications that makes them ephemeral. The print communications of the past were constructed in a way that impelled people to perceive a continuity between the present, the past, and the future, mirroring the chronology of events as laid out in, say, a novel or historical treatise. In the age of digital communications, we may have lost some of this sense, since communications are fragmented, reproduced constantly and renewed quickly. As a result, we feel suspended nowhere and everywhere, which, McLuhan asserted, is the same form of consciousness experienced by those who lived primarily in oral (and pictogram-using) cultures, whose sense of history and reality was more episodic than chronologically linear.

Emoji fall under this paradigm, where history is perceived more and more as a set of episodes than a concatenated linear set of events, much like the images in a picture book or a comic book. Although they surfaced as a means to enhance broader comprehension of written texts in the global village, culturally stylized forms have nonetheless emerged for various reasons (as we saw earlier). Recall the example of facial emoji. By and large, the creators of the smileys have attempted to make them as culturally neutral as possible. But, almost right after their introduction into common usage, new emoji were constructed that embedded culturally based meanings, either explicitly or unwittingly. The nerd and sleuth emoji, for example, require understanding what a “nerd” or “sleuth” implies in culture-specific ways. These would make absolutely no sense to, say, a society living outside the realm of modern urbanized popular cultures from which these signs emerged. It is perhaps more accurate to say that some emoji are higher on a “universality scale” than others. The slightly frowning , smiling , and face with rolling eyes emoji can be located higher up on the scale; others, such as the robot face , are likely to have “mid-scale” comprehensibility; and others still, such as the live long and prosper emoji , are likely to have a lower-scale comprehensibility. The last emoji alludes to a pop culture referent – the 1960s Star Trek program on American network television, where it surfaced as the Vulcan peace sign. Although the sign has become somewhat of a common symbol in many parts of the world, its interpretation is constrained by various variables, such as the age of the emoji users and their geo-historical background. As such, emoji might represent a langue all its own, to use Reference SaussureSaussure’s (1916) term, and one that is constantly evolving, accruing subtle meanings known only to seasoned users. But it is a different kind of langue in that it has eliminated the traditional distinction between separate verbal and visual modes of representation and communication – that is, between verbality and visuality.

But it is exactly the emoji langue’s connection and relevance to the modern world that can play a key role in education at all levels. Emoji allow for a coordination of ideas expressed verbally and visually, and thus for a semioliterate organization that integrates visual characters and verbal ideas expressed cohesively, subject to cultural, subjective, and technological variance factors. Because emoji are structures that can be applied to any message on the spot, they are highly effective for allowing the activation of agency and alterity factors that are now practically de rigueur in the multicultural, multiracial, and developmentally differentiated classrooms for real learning to unfold. The dangers of overusing or misusing emoji is part of a broader pedagogical discussion that can be elaborated (Chapter 6); overall, there is little doubt that they can level the educational playing field for all students.

In sum, emoji have made hybrid literacy both a fact and an educational imperative. The argument will be made that such literacy is a point of departure for imparting academic literacy (Chapter 2) and a means for facilitating healthcare communications (Chapter 4) as well as the education of healthcare professionals (Chapter 5). In the print age, with its emphasis on linear progression, humanity constructed linear-narrative models of history that suggested that the future was infinitely far away and that personal and social evolution was involved in finding a path toward it; in the Internet age, there is no comparable cognitive linearity; there is mainly connectivity that makes us feel situated within a network structure.

So, as a final introductory thought about emoji, one can legitimately ask: Are emoji a revolution in human communication or a passing trend? Will they be around in, say, 2089? Emoji in themselves are not revolutionary, given that they retrieve pictographic forms of writing in a new technologically standardized way. So, they will likely morph into something else or disappear entirely as the technology changes. The Oxford Dictionary may have been right after all, sensing that emoji captured a moment in time. It could also well be that emoji bear an unconscious desire that reflects the times – the need to render communication as friendly as possible. In conducting interviews with students, on the part of all three authors of this book, the overall view that emoji convey a “sunny” tone to human interaction surfaced commonly. It is no accident that the most common emoji bear the color of the sun.