Introduction
Responding to ‘alarming macro-trends’ (Bhaskar Reference Bhaskar2013, 3) in historical methods of media distribution brought on in part by the ‘disrup[tion] and disintermedi[ation] … of traditional value chains’ (Murray and Squires Reference Murray and Squires2013, 3), the study of transmedia initially foregrounded questions of definition and commercial potential. Scholars like Henry Jenkins saw an opportunity in what he called media convergence: ‘a situation in which multiple media systems coexist and where media content flows fluidly across them’ (Reference Jenkins2006, 282), and transmedia as “a process where integral elements of a fiction get dispersed systematically across multiple delivery channels for the purpose of creating a unified and coordinated entertainment experience” (Reference Jenkins2007). While there was significant parallel interest in the kinds of stories transmedia might afford, the focus primarily rested on the what of transmedia; when Marsha Kinder first coined the term, for example, she used it to describe the ‘supersystems’ constructed by corporate synergies and multinational corporate mergers (Reference Kinder1991, 38).
A subsequent decline in transmedia scholarship since its peak around 2010 has coincided with a shifting of perspective from the production of transmedia to its reception. This focus in itself is nothing new: in addition to being an established part of media and literary theory, the significance of audience and reception was an omnipresent component in earlier transmedia thought. A self-confessed acafan (an academic who identifies themselves as part of a fandom) Henry Jenkins was instrumental in establishing said fans as active and participatory consumers of media products, through his foundational work on textual poaching (Reference Jenkins1992).
Ownership of these textual microterritories (Lévi Reference Lévy1997), however, predominantly framed transmedia as a textual concern: whether, for example, a fan work forms a legitimate part of a larger transmedia world, or if it is important to ‘maintain clear distinctions between canon and fanon’ (Jenkins Reference Jenkins, Freeman and Gambarato2019: xxix). Intriguing recent studies have returned more fruitfully to the question of audience: one paper sought to resolve the challenge of how to close-read a transmedia story by identifying structural patterns, for example (Javanshir et al., Reference Javanshir, Carroll and Millard2020). Such approaches, however, do not address the way in which individuals themselves reconstruct the narrative through their memories; the aforementioned paper is primarily interested in the development of a new taxonomy for classifying transmedia experiences, for example.
It is this latter component of the transmedia phenomenon – not how we read such stories but how we remember them – that this paper seeks to address, a gap in existing representations of transmedia effects. Rather than focusing on the preparation of a transmedia narrative for purposes of distribution and consumption, it explores the way we spontaneously remember content previously presented across multiple media and other ‘immediate’ spaces. It reveals the way in which the same vignette or moment presented in different media is ‘remixed’ and revised in the act of remembering.
The focus within transmedia scholarship tends to be on either collective memory or how we engage with the past, or the relative canonicity of fan recall and its relevance in a commercial system. Rather than cultural memory, which is informed by the sum of representations across media and cultures, our focus is the individual subject's bespoke co-location of discrete representations into a gestalt, a medium-agnostic structure – with a specific focus on the individual's co-location of distinct representations of the same media event. The effect of transmedia on this aspect of reader response, e.g., memory and recall, has yet to be fully understood and, therefore, encoded in actionable structures. To that end, the first section offers the initial findings of a recent study undertaken by one of the authors. Through semi-structured anonymous interviews, 190 participants were asked to recall works of literature which they remembered best. The outcome speaks to the intrinsically transmedial nature of memory, and its relationship with narrative. This contribution is a first report, providing an initial set of definitions and hypotheses for what we believe to be a necessary targeted study, and intended to provide context for further development by future scholars.
The research was conducted anonymously in the form of semi-structured interviews involving 190 readers, split about evenly between two countries, Croatia (90 interviews in 2011) and USA (100 interviews in 2016). While the limitations and implications of cultural differences among the two reader populations are outside the scope of this paper, the way in which intertextual and transmedial references were made in both countries showed no marked differences in how they accessed both print and film media. Therefore, among the ‘many possible ways to try to bridge the divide between individual and collective memory’ (Schacter et al., Reference Schacter, Gutchess, Kensinger, Boyer and Wertsch2009, 83), or between memories in the head and in the wild (Barnier and Hoskins Reference Barnier and Hoskins2018), we choose to focus on the models and media implications which may best describe and contextualise the principal observation in terms of everyday memory application.
How does the medium of reception affect the quality and nature of recall? Do similar notions presented across multiple platforms supplement, supersede, or integrate with one another? Does a ‘recall hierarchy’ exist that could ensure favoured content surfaces more readily? Do current models reflect the densely intertextual nature of remembering? To illuminate this and other interrelated ideas, we begin by exploring what the authors have called spontaneous transmedia co-location (STC). We use spontaneous in the sense of being in the moment, to highlight that the interviewees elected to employ the multiple media references themselves, as well as the initial literary titles; we use co-location to describe memories of the same content in different media occurring together in a particular act of memory retrieval.
‘The relevance of the media for individual and social forms of memory,’ notes Zierold (Reference Zierold, Erll and Nünning2009, 399), ‘is widely acknowledged by representatives of memory studies.’ This paper sits at the intersection of several areas – mostly explicitly transmedia, memory, and literary studies – which all have bearing on the unexpected phenomena identified through the initial study. We have sought to acknowledge adjacent disciplines beyond our own, but recognise that our focus is on the impact of this newly found phenomenon on existing models of reading. Its implications for wider memory studies represent a strong opportunity for future research, though effort has been made to acknowledge existing work in this area.
‘Considering the various contrary tendencies and the trend towards a pluralisation of processes of remembrances in society,’ continues Zierold in the same volume (405-6), ‘the connection between the media and memory cannot be described only in a theoretical manner but must also be observed in detail empirically. There is a demand for studies which substantiate the initial theoretical concepts empirically, which examine processes of remembrance under the conditions of modern media systems.’ Readers are still considered by many scholars to be ‘the dark matter of the literary universe’ (Andersen et al. Reference Andersen, Kjerkegaard and Stougaard Pedersen2021, 134), so interdisciplinary work such as ours can hopefully begin to illuminate it, while at the same time answering the need for empirical detail pointed out by Zierold.
Spontaneous transmedia co-location
A recent study conducted by one of the authors aimed at providing some empirical details on how non-professional readers remember works of literature. By non-professional readers, we refer to those for whom close critical reading of literature is not a necessary or fundamental part of their working life i.e. the roughly 99 per cent of ‘ordinary’ readers who are not editors, journalists, academics, writers, or members of any parallel profession. The subjects of the research were, therefore, what Merve Emre provocatively identifies as the ‘mass of bad readers’ and ‘a kind of irritating background noise; always already present and unworthy of any serious or systematic consideration’ (Reference Emre2017, 2–3). This research considers such readers and their memories seriously and systematically, however, straddling the broad disciplinary fields of reader reception theory and memory studies, while aiming to empirically specify instances of what was also recently described as the ‘unsystematic and disorganised’ nature of ‘[u]ncritical modes of reading’ (Warner Reference Warner2012, 15), or a ‘postcritical reading’ which ‘slices across [the] dichotomy of skeptical detachment versus naive attachment’ (Felski Reference Felski2020, 135).
Although the interviews were conducted in two different languages (Croatian and English), the two reader groups may be treated as one, and the results of the analysis may be observed together, as the methodology was nearly identical for both of them. The respondents were recruited in person by the sole interviewer (and a paper co-author) at sites linked to literary reading, such as libraries, book stores, and book cafés. After being introduced with the aims of the interview and the Institutional Review Board (IRB) procedure that approved it, the interviewees gave their oral consent to having the conversation recorded (audio only), which then proceeded in a mostly informal fashion, with the aim of obtaining information of high ecological validity (i.e., resembling a casual conversation about books). The question list differed slightly between the two countries, where it was conducted in Croatian and English, respectively, but it always consisted of five sets of questions, with the first and last set being fixed in their positions, and the others being interchangeable. Set I was the preparatory one (choosing three to five books to discuss), set II was about the content and meaning of the texts, set III about facts on the text and paratext, set IV about the circumstances (environment) of reading, while set V consisted of concluding remarks by the interviewer and interviewee (for more details, see Škopljanac Reference Škopljanac2023).
All of the interviews were conducted anonymously with subjects aged 20 and above, and their average age was 40. No other information about the respondents was consistently collected, except for their sex: 105 females and 85 males took part in the interviews, which is broadly indicative of the trend of more women reading literary works in both countries.
The interviews on average lasted about 25 minutes, ranging between 15 and 90 minutes. The respondents were first asked to select no less than three works of literature they remembered well, and then they were asked a set of questions designed to probe what exactly they remembered the most, and correspondingly, what they forgot most readily. In that respect, the study was designed to an extent as a reconfiguration of what constitutes literature, along similar lines to oral history investigations conducted previously (Lyons and Taksa Reference Lyons and Taksa1992) and subsequently (Trower Reference Trower2020) in other English-speaking countries. Although very few such studies have been made, the Trower study is comparable to the present one in its methodology and results, particularly when it comes to the type of responses elicited:
‘To use psychological terminology, when readers are interviewed about their memories of fiction, the type of memory that seems most operational is usually ‘episodic’ (concerning autobiographical experiences that can be explicitly stated) rather than ‘semantic’ (encompassing the ‘storage of words and meanings’). Readers tend more readily to remember experiences of reading novels, in other words, than the content of the novels themselves’ (Trower Reference Trower2020, 271).
Therefore, even the answers which were elicited mostly by using prompts about semantic memory (e.g., ‘What is the book about?’) resulted in episodic memories in the majority of answers, at least where specific titles were concerned: roughly 70 per cent of the 190 readers, according to the analysis which was performed using open coding by the interviewer.
The titular effect was initially observed during the subsequent close reading of the interviews by the study author, a partial response to experimenter bias since this effect was not a part of the original study design in any way. The unexpectedness of the result is also why a number of other potential issues were not taken into consideration during the interviews, such as different forms of the same content, e.g. a poem printed in a book vs. shown on a computer screen. Although the issue of different affordances of media (see Magliano et al. Reference Magliano, Loschky, Clinton, Larson, Miller, Cutting and McCardle2013) and the modalities of their reception have been receiving deserved attention (Turner and Felisberti Reference Turner and Felisberti2018, 265), this study's approach was unidimensional and exploratory in comparison, dealing with different facets of readers’ memory. That is to say, data was gathered only sporadically about substrates, or the physical form of the text and the paratext (see Mangen and van der Weel Reference Mangen and van der Weel2016), like the smell of the pages or the look of the front cover. However, questions about the substrate (e.g. ‘What did the book look like?’) never elicited answers about other media and its potential effects, so that issue is not considered relevant for the present paper, unlike in other studies where the modality of reading written text is the central issue (see Kosch et al. Reference Kosch, Stocker, Schwabe and Boomgaarden2021, which devotes a short section to ‘memory and retention’).
Alongside these, there are several other issues which might be of relevance to transmediality and memory studies more generally, such as trans-symbolic comprehension (Steciuch et al. Reference Steciuch, Millis and Kopatich2023), misattribution and false memories (Schacter Reference Schacter2001: 5, 8 et passim), self-referential encoding (Carson et al. Reference Carson, Murphy, Moscovitch and Rosenbaum2016), (reading) metacognition (Trasmundi et al. Reference Trasmundi, Kokkola, Schilhab and Mangen2021, 7), and so on. However, they will have to remain outside the scope of this study and paper due to, respectively, their qualitative nature and focus on describing the titular effect, and not testing for it.
Having cleared up what the study was not about, it is still striking to note how the very first interviews clearly demonstrated a transmedial effect present in the way respondents recalled works they considered most memorable – in particular with regard to film. This blending perhaps reflects the wider migration of narrative to visual media (Heise Reference Heise2017, 280), and the parallel cognitive effect of the literary work's paucity of ‘directly perceptible attributes’ compared to ‘paintings, films, and music’ (Mangen and Kuiken Reference Mangen and Kuiken2014, 151).
This sort of blending, specifically around literary texts and films, has already been observed (in passing) in various research contexts: distant readings of Victorian literature (Bourrier and Thelwall Reference Bourrier and Thelwall2020, 22); anecdotal evidence from a scholar dealing with mental representations of literature (Elfenbein Reference Elfenbein2020, 247); cultural studies research on Australian non-professional readers (Collinson Reference Collinson2009). The latter study attributes this intertextual effect to both ‘the media literacy of this particular reader’ who discussed the film adaptation of Alice Walker's The Color Purple, and to ‘the complex ‘synergy’ between different cultural forms across the mediascape’ (115).
Clearly, then, a disposition exists for contemporary readers to reference films while recalling literary texts. Although such comments constitute a minority, it is a sizable one: 14 out of 90 (16 per cent) Croatian interviewees made such references, with one person additionally referring to a theatre play patterned after a book they had read. Similarly, 36 out of 100 (29 per cent) American interviewees made such unsolicited comments, with one person also referring to a video game patterned after a book they had read. Combining both studies, this amounts to a quarter of the sample: Twenty-six per cent of the 190 interviewees articulated the titular effect. A total of 23 of them were female, 27 male; their average age was 46, with a standard deviation of 19. Most interviewees referred to only one movie per interview, though some mentioned multiple movies instead. The complete 50 reader sample, consisting of all the respondents articulating moments of STC, includes about 50 movies and TV shows, as well as about 150 books, depending on how the references are counted (e.g., multiple adaptations of the same novel). Twenty of the extracts are presented in Supplementary Appendix 1, with the Croatian ones translated into English by one of the authors. Every quote from the interviewees is preceded by the title and author of the text [in square brackets] which led to the film recollections, and followed by a tag (in round brackets) indicating: the location of the interview (HR = Croatia, US = United States of America), the interviewee's code number, gender, and age. To illustrate the findings, here are five selected extracts:
Extract 1. [The Lord of the Rings, J. R. R. Tolkien] ‘Let's say that part when they are in the mines of Moria, when they are attacked by those goblins, or whatever they are already called. And then those drums are heard. [Asked if he watched the film] Yes, that's why I stopped to think for a bit … I'm a little confused about what I remember from the movie, and what I remember from the book’ (HR-1, M, 25).
Extract 9. [Lolita, Vladimir Nabokov] ‘Now, it's hard to say whether it's from the novel or from several versions of the film which I watched ( … ) I think it's from the novel, there's a scene in which … I think that … She got something in her eye, and then he runs his tongue over her eyeball to get it out. It's a scene, quite unusual, that I remember. I'll say again, I think it's from the novel, I don't think it's there in the film version’ (HR-78, F, 33).
Extract 10. [A Clockwork Orange, Anthony Burgess] ‘So there is also a movie and I'm sort of blending the … I'm trying not to mix the two up. ( … ) When I think of it, I think of the line in the book and the actor saying it’ (US-77, M, 21).
Extract 15. [U registraturi, Ante Kovačić] ‘That show was on TV, then I looked for the book as well. I don't know what year it could have been, ‘65, ‘70? I know that Ljubica Jović played in it. Maybe [the male protagonist] was played by Rade Šerbedžija, I don't know? The show itself was nice, so I wanted to take a look at the book, ( … ) to compare them a bit. Then I deliberately searched for that book.’ (HR-30, F, 67).
Extract 19. [Of Human Bondage, W. Somerset Maugham] ‘His persona was portrayed in Hollywood movies that were movie renditions of some of his books. And I think Hollywood did both Of Human Bondage and The Razor's Edge. I know they did The Razor's Edge at least, in fact, twice. ( … ) So I have this image now – I don't remember the name of the male actor that portrayed him, he was a very well known actor of his time, long dead now – but he sort of is my image of Somerset Maugham even though he [has] no resemblance to him whatsoever.’ (US-6, M, 80).
The initial analysis of these unexpected findings suggested that transmedia co-location is the result of several phenomena, not just one. We now introduce our account of the distinct phenomena we have identified – five in total – that range from the emergence of unsolicited correlations to mixed-up connections with original sources and merging of different media experiences around pivotal narrative moments.
The below uses the term entanglement for its beneficial dual meaning – both in the sense of being intricately bound together (as in quantum entanglement) and the more analog, tangled but still distinct entangling of twine. This recognition of both the closeness and messiness of this binding together serves our definitions well.
1. spontaneous entanglements: unprompted free association between media content resulting from personal experience of works.
2. fuzzy connections confusing the media-content origins of specific memories.
3. media complementarity: unprompted value assessment of source medium resulting from dissonance in memories.
4. sensory blending of visual and auditory inputs.
5. scaffolding of experiences of narrative moments included in different content-media into a unique mental representation.
Supplementary Table 1 summarises the relevance of Appendix extracts concerning the five phenomena.
These phenomena are likely part of the same transmedia co-location dynamic. However, the testimonies of reading experience we collected do not present consistent evidence of all five simultaneously at work. The extracts in the Appendix are a sample of first-hand evidence meant to accompany the presentation of this aspect of readers’ memory and the subsequent discussion of the main issues characterising transmedia narratives.
Gratifyingly, the very first interviews conducted evidence the two most common examples of this phenomenon, as they are not mutually exclusive and readers would sometimes manifest several of them (see Supplementary Appendix 1 – Interview Extracts). In both cases, the respondents recalled memories about corresponding films, despite being asked only to talk about the written work. Even though there was additional prompting in the first interview, the context makes it obvious that HR-1 thought of the adaptation of the book without being asked to do so. This is also clear in the second answer and brings up the first major point about these recollections: they occur spontaneously, disregarding the textual framing of the original source material. That is, even though the respondents were trying to recall pertinent semantic memories about the written text(s), they also spontaneously retrieved semantic content specific to the film(s) made after the books, with both seeming to be inextricably connected. The second point is that precisely this seamless transmedial recall creates sequence confusion on behalf of the readers; in short, they are unsure which is the proverbial chicken and which is the egg, or which medium they engaged with first.
This confusion also seemed to create minor tension with some readers, which in some cases led to them offering critical judgement of the entangled works. Readers would readily accompany recollection of both a written text and film with a value statement comparing the two works, or even the two media. In those cases, it was always asserted that the written text is superior.
In addition to the effortless recall already identified, these extracts demonstrate deficiencies in what psychologists refer to as source monitoring (Johnson et al. Reference Johnson, Hashtroudi and Lindsay1993), or determining where specific memory content is derived from. Readers can make out some differences, especially paratextual aspects (names of book authors, movie directors) and their critical judgement – but not to a large extent, and in one extreme case not at all (one respondent was unsure whether she had even read the Wuthering Heights, or if she was just able to recall ‘scenes from the movie’).
This also allows us to bring up another point, as exemplified in the first (HR-1) and third (US-77) examples quoted: that these memories seem to blend both visual and auditory inputs, as both interviewees remember hearing specific sounds from the films (the drums and the actor's voice, respectively). While an investigation of exactly how (and how often) readers mentalise sound indicated in a text while reading silently is beyond the scope of this paper, the pronounced auditory aspect of the film is evidently and readily inserted into a textual memory in these cases.
Discussion
As this initial study has indicated, the agnostic fluidity of remembering is a complex process, paradoxically comfortable with both the seamless integration of sense-experience from disparate media into existing memory, and with the hierarchical organising of dissonant content (with associated value judgement in some cases). While not entirely new or unexpected in a broader theoretical context, these findings represent an opportunity for improved formal representation and classification of content and media experience.
Transmedia and the wider study of memory have an interest in how connectivity and the digital reshape our relationship with media. The idea of convergence – ‘the flow of content across multiple media platforms, the cooperation between multiple media industries, and the migratory behaviour of media audiences who would go almost anywhere in search of the kinds of entertainment experiences they wanted’ (Jenkins Reference Jenkins2006) – found in transmedia has parallels with the connective turn in memory studies – ‘the sudden abundance, pervasiveness, and immediacy of digital media, communication networks and archives’ (Hoskins Reference Hoskins2011). Both interrogate the impact of this flow across networks and of this profound, reshaping connectivity. In each case, there is a question around immediacy, whether the participant ‘forgets the presence of the medium’ (Bolter and Grusin Reference Bolter and Grusin2000, 272) in deference to content. This survey evidences the fluid, complex process by which participants themselves navigated this specific question.
Our remaining discussion is an attempt to ground transmedia co-location within existing literature on reading, reflecting the specific scholarship of the authors. Scholars of interdisciplinary research (Klein Reference Klein2005; Boix Mansilla Reference Boix Mansilla, Frodeman, Klein and Mitcham2010) note the limitations generated by the lack of a shared standard, the ‘cognitive obstacles’ represented by domain specificity (MacLeod Reference MacLeod2018). The study of transmedia co-location, its correct definition and scoping, requires a more focussed future study properly designed around this specific phenomenon, by scholars better able to grasp its impact for their respective fields. As noted above, this contribution is a first report, providing an initial set of definitions and hypotheses for what we believe to be a necessary targeted study.
The relationship between memory and transmedia has seen discussion elsewhere. Remembered events, writes Astrid Erll (Reference Erll, Erll and Nünning2008, 392), are ‘transmedial phenomena, that is, their representation is not tied to one specific medium. Therefore, they can be represented across the spectrum of available media. And this is precisely what creates a powerful site of memory.’ A similar sentiment is expressed by scholar of transmedia memory Colin Harvey. Describing the configurative practice required for engaging with media as being ‘interpolated by memory,’ Harvey argues that not only is the viewer, player, or reader ‘required to configure the medium in question in order to understand the piece of narrative on its own terms, but to engage with the work transmedially’ by recalling narrative elements as presented in other media (Harvey Reference Harvey2015, 3). Work on reading deals extensively with the impact of background knowledge, including personal experience, on narrative experience – Richard Gerrig and Micah Mumper's exploration of the way readers’ lives affect narrative experiences (Reference Gerrig, Mumper, Burke and Troscianko2017, 239–245), for one example among many. Discussing our integration of personal experience as we interact with a narrative world, these authors note that we draw on domains of personal memory (to which we might add our knowledge of the narrative world of the work) to interpret new works, with discontinuous elements requiring integration or a recognition of co-location.
In her paper We Have Always Wanted More, Susana Tosca identifies our desire for transmedia as arising from ‘a realization about a lack’ (Reference Tosca2015, 37). Passion for the storyworld drives both fanfiction expansion of an existing universe, and our yearning to fill out those details we feel are absent. Readers of Lord of the Rings, for example, suffered decades of speculating on Aragorn's appearance. For those who found Viggo Mortensen convincing casting, his physical appearance may now be amalgamated with that of the print character; for others, who consider him unconvincing (‘too clean’ according to the partner of one co-author), we may switch to value judgement (media complementarity) about the weakness of the cinematic medium for representing beloved works. Such comparisons may not be wholly agnostic to wider cultural contexts, of course: Harvey notes that certain media is ‘accorded more authenticity’ (Reference Harvey2015, 3) by fans, citing the relative value afforded to audiobooks vs televisual representations of Doctor Who. It is always important, as Paul Duguid rightly says, to think ‘not idealistically about information, but materially’ (Reference Duguid and Nunberg1996).
The scaffolding effect we identified above is likely connected to the relationship between single-content and transmedia experiences being a multi-scalar one; following our first encounter with a narrative property, successive elements can be integrated into our existing understanding. In this view, engagement with adaptations of existing works can be seen as an iterative revision to an existing mental structure, building from our first encounter with a transmedia property. Consider, for example, the extent to which discussion about what Star Wars ‘is’ relies on a foundational engagement with the original 1970s films.
While the focus of this essay is on content or media experiences – primarily reflecting the way in which the different flavours of STC manifested in interviews – there are fruitful potential avenues in looking at both ‘canon’ but also endorsed paratexts (see Jonathan Gray's Reference Gray2010a, Reference Gray2010b work on promos, for example, considering how fans integrate marketing or promotional images) and the extensive literature within transmedia on convergence and fan content (see Jenkins Reference Jenkins2006 among others, considering whether fanfiction or community narratives integrate into recollection).
Such revisions are constrained by the sensorial affordances of specific content-media and may supplement the existing mental images developed during previous interactions with the story world. Each discrete medium has, as Erll puts it, ‘its specific way of remembering and will leave its trace on the memory it creates’ (Reference Erll, Erll and Nünning2008, 389). Rather than being narrowed, however, audiences can actively integrate even aberrant or dissonant elements into their overall understanding. It is not necessary to deny the existence of Viggo Mortensen's performance to maintain a consistent internal picture of Lord of the Rings, for example; indeed for fans, railing against this portrayal becomes part of the pleasure of the text. Extratextual elements can be approached similarly: an objection to the politics of J K Rowling, to give another example, can be integrated into an understanding of Harry Potter lore without requiring the obfuscation or rejection of the whole work (see also negotiated reading in Hall Reference Hall1973; Gerrig and Mumper Reference Gerrig, Mumper, Burke and Troscianko2017).
Implications for approaches to reading
In his 1974 work The Implied Reader, literary scholar Wolfgang Iser notes that while interpretations may differ between readers, such interpretations exist only ‘within the limits imposed by the written as opposed to the unwritten text’ (287). A transmedia space, however, is an evolving system that begins with one first work, content, and medium, but views this as potentially one facet of a larger narrative space. In this view, it is possible to reframe a simple content experience (i.e. watching a film) as a special case of transmedia experience – one in which, by choice or availability of media, the user-reader engages with only one work (content and medium). Following this logic, a multi-modal experience can be seen by contrast as an engagement with multiple medium or content-medium combinations, e.g. reading both book and e-book, or reading the book and watching a movie.
From its initial state, a transmedia space may subsequently grow by multiplying distribution media (making stories available on other platforms) or through new interconnected works sharing the same narrative elements (the aforementioned Doctor Who audiobooks and movies). Our definition of the ‘unwritten text’ thus becomes more complex. Since each first work (that is, the first work encountered by a reader) can in principle generate an arbitrary amount of potential content, and that content can hypothetically be distributed across an arbitrary number of potential media, transmedia spaces warrant a more complex articulation of the relationship between content and medium (see Supplementary Table 2).
To accommodate the richness of a full transmedia experience, models of reading must equally be able to accommodate individual systems of explicit–implicit reference, alongside this traversal of multiple works. Such ‘intricate entanglements’ (Freeman and Gambarato Reference Freeman and Gambarato2018, 407) call for a more sophisticated interrogation of our approach to intertextuality and transmedia, the densely ‘intertwingled’ nature of recall. This term, coined by Ted Nelson and taken up by subsequent scholars of information architecture, helps us understand that knowledge does not fit into neat ontological categories, but is densely and complexly interconnected. ‘In designing taxonomies and vocabularies,’ warns Peter Morville in his Reference Morville2014 work on the subject, ‘we serve as architects of understanding’ (46). Connection is a deeply personal, sometimes arbitrary process, and has a complex relationship with the largely legal distinctions between different media, or the privileged connections generated by the practice of authorship.
As an act, our journey within a transmedia space is the result of negotiations between a reader and the community of practice which they belong to or interact with. Furthermore, such ‘non-diegetic’ interactions (enveloping the narrative space) are legitimate material for experiences in the frame of a transmedia space. For instance, discussions on consistencies and inconsistencies, complementarity and alternatives between contents and fan theories both drive a transmedia audience, and are raw material for the expansion of a transmedia space. Any description of the underlying network of associations should be able to recognise (a) these elements of negotiation (e.g., inconsistency, parallel alternative pathways), and (b) a different ‘level of authority’ in terms of being established by author(s) or influential figures in the narrative space due to, say, verified or unverified hypotheses and a form of ‘narrative veracity’ (see Supplementary Table 3).
As discussed, the experience of a transmedia space is qualitatively different from the experience of a single content or medium. The characterisation of such experience is, however, difficult to achieve from both an ontological and practical point of view. Indeed, while content and media present a visible structure and mode of interconnection, the transmedia space is simultaneously a site for numerous implicit, vague, or fuzzy connections. Furthermore, as for narrative, the tension between gaps and contribution of the reader has a central role in the experience. However, for a transmedia space, this tension is also a key driver for generating the space itself, in the form of a ‘cultural memory’ complementing the visible traces of the lore (i.e. work(s) and adaptations).
Conclusions and future work
This paper explored the notion of STC, defined by five initial properties. The reported study was not designed to study this specific dynamic – transmedia co-location – and, therefore, we cannot provide a final statement about its underlying mechanisms and their potential interconnections.
As a hypothesis, we provided a working definition of the five phenomena based on a simplified model of transmedia experience: [A] interaction (senses and cognition), [B] response (first elaboration of emotions, memory, and evaluation), and [C] reception (re-elaborations of sentiment and evaluation mediated by time and other following experiences). In these terms, the five phenomena can be defined as the following alterations (see Figure 1):
(1) spontaneous entanglements is the insertion of a content-medium reception within the reception of a second content-medium
(2) fuzzy connections is cross-pollination of the experience of a different content-medium in the response and/or reception of a content medium
(3) media complementarity is the dominance of an overarching indivisible combined reception over the reception of individual content-media
(4) sensory blending is the dominance of an ex-post mental reconstruction of multimedia experience over the originals as a new source for response and reception of transmedia
(5) scaffolding is the dominance of an ex-post reconstructed response of transmedia as the new source for reception.
Figure 1. Evidences of transmedia co-location are found in the extracts in Supplementary Appendix 1.
In this view, transmedia co-location is the breaking of the causal chain connecting interaction, response, and reception of distinct forms of transmedia.
These working definitions should help in unpacking other hypotheses about the nature of transmedia co-location. For instance, the effect of time seems to be relevant in the creation of, e.g., fuzzy connections. The timing between the experience of medium-contents may play a role in terms of pre-elaboration of an experience to be fuzzy-mixed in another. For example, how fresh the memory of a movie should be to influence reading a book? This consideration seems to support a more general consideration about how difficult it is to capture and study transmedia co-location.
This work intersects with narratology, audience studies, and memory studies in a variety of ways that encourage future study, particularly in relation to interactive narrative. Video games, for example, are profoundly intertextual multimedia artefacts which, deliberately or not, reference work in other media. Properly understanding the way in which the medium of transmittal affects the integration of narrative content into pre-existing memory structures represents a unique opportunity both for scholars of interactive narrative, for whom there is a limited language for discussing this specific form of connectivity, and those whose role is to determine to which platform-specific content should be addressed.
As outlined in the introduction, transmedia emerged as a broadly commercial phenomenon, and there are opportunities to reintegrate these findings into existing commercial processes. Weaknesses in source-monitoring and the development of a memory kernel can be integrated into the serial nature of transmedia properties. By approaching specific events iteratively, returning to them in subsequent works in different media, a memory may be ‘revised’ to greater prominence. If continuity demands an event to be present at a specific point, for example, a canny producer might present it as written text, filling in the moment cinematically when budget and time allow it.
This approach of re-reading – not returning to the same content, but exploring it – should be familiar to scholars of hypermedia and hypertext fiction. Significant previous work in that field (Yellowlees-Douglas Reference Yellowlees-Douglas2003; Galef Reference Galef1998; Mitchell and McGee Reference Mitchell and McGee2012) has explored the benefits of re-reading, and the difference between returning to a work that remains unchanged, and the work that yields new lexia with each reading. This study in part reverses this notion: rather than re-reading being the discovery of new content that supplements current understanding, new experiences are integrated into existing memory structures in a complex way that is hard to define, but which seems to be the default mode of reception.
By way of comparison, recent work on webcomics has highlighted the role of ‘transmedia infrastructures’ (i.e. based on emerging practices and loose technical connections) in creating a transmedia experience across platforms and involving multiple actors in different roles (Antonini et al. Reference Antonini, Brooker and Benatti2020). The dimensions of media analysis should extend to how media support the transition between phases of the content lifecycle and the engagement of different actors at each stage. While we could consider them as external aspects of content experience, these ‘non-diegetic’ interactions complement and merge with classical ‘diegetic’ interactions with the narrative. An exploration of the non-diegetic memories of individual participants (the impact that the casting of an unlikely actor in a film has on the response to the written work, for example) might help identify additional spontaneous memory entanglements.
Above all, however, this paper advocates for the repositioning of the reader as the locus of enquiry for models of reading and related explorations of transmedia and memory. Locating the study of transmedia in industrial practice or material connection ignores that meaning, to return to Wolfgang Iser, resides with the reader. Attempts to understand association, intertextuality, and connection must accommodate not just those connections that are conveniently, positively material. We must acknowledge the messy, often contradictory nature of the way readers engage with and recall works. This refocusing on readers would represent a fruitful new avenue that might help revitalise transmedia scholarship, bringing it closer to the reception studies that are (to our thinking) its natural home.
‘Transmedia storytelling,’ writes Harvey (Reference Harvey2015, 38), ‘is all about memory.’ Recall of the various elements that make up a storyworld is one aspect of this, but equally important is the way in which we encode these memories, integrating and amalgamating events as we engage with them across different media. Narrative moments presented across multiple media exist under the jurisdiction of a complex network of transmedial and intermedial recollections and rememberings. This contribution affords an opportunity to better understand the benefits of both close reading and wider studies of reception, and how they might provide insight into the way in which memory is constructed across diegetic, visual, and textual spaces.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/mem.2023.13.
Data availability statement
The data that support the findings of this study are available from Lovro Škopljanac, one of the authors ([email protected]). Restrictions apply to the availability of these data, which were collected after undergoing an IRB at Penn State University, USA.
Funding statement
This work was partially supported by Reading Europe – Advanced Data Investigation Tool (READ-IT), a project funded by the JPI Cultural Heritage under the European Union Horizon 2020 Research and Innovation programme (grant agreement No 699523).
This work was partially supported by the Reading Europe Advanced Data Investigation Tool, a project funded by the AHRC/UKRI (grant agreement AH/S000410/1). Data and output are in open access licenses and available on the project repository (https://ordo.open.ac.uk/account/home#/projects/88856) and GitHub page (https://github.com/eureadit). This work was partially supported by the Croatian Science Foundation under the Project UIP-2020-02-2430. None of the funders or supporters had any role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Dr Alessio Antonini is a research fellow at the Knowledge Media Institute and the lead of the Knowledge Design team. His research focuses on the design of intelligent and interactive media, research infrastructures for the Humanities, and AI-enabled innovation for community and personal wellbeing.
Dr Sam Brooker is an Associate Professor of Digital Communications. He specialises in digital communication and his research explores the relationship between digital technologies and theories of literature and culture. He is a regular contributor to ACM Hypertext and Social Media, a member of the PRCA International University Advisory Group, and an academic reviewer for numerous journals.
Dr Lovro Škopljanac is an Assistant Professor of Comparative Literature He obtained degrees in Comparative Literature, English Language and Literature, Japanese Studies and Conference Interpreting, and a PhD in Comparative Literature. His research interests are Comparative Literary Studies, Contemporary Literary Subjects (readers), Cognitive Literary Studies, and Computational Literary Studies (4xCLS).
Open access
