Hostname: page-component-586b7cd67f-t8hqh Total loading time: 0 Render date: 2024-11-29T07:58:22.007Z Has data issue: false hasContentIssue false
  • English
  • Français

A Linked Open Data model for describing comic book sequences: Exploring semantic enrichment opportunities with graphic medicine

Published online by Cambridge University Press:  12 July 2023

Sean Petiya
Sean Petiya*
Affiliation:
Adjunct Instructor School of Information Kent State University 800 E. Summit St. Kent, OH 44242 USA Email: [email protected]

Abstract

Applying a Linked Open Data (LOD) approach to modeling the visual structure and content of comic books and graphic novels enables the description of these works to be enhanced through the process of semantic enrichment. This strategy may be particularly impactful for graphic medicine, a non-exclusive genre of comics that communicate medical and healthcare information, including personal stories of illness. However, the metadata for these works may lack references to healthcare-related vocabulary, thesauri or ontology that would more precisely describe their contents. This material may include pages and panels that illustrate specific medical topics, such as symptoms, side-effects or treatments — subjects that often overlap with other healthcare challenges, including mental health and illness. Exploring an LOD approach for describing comics content may potentially enhance the discoverability of this material and its ability to be remixed and reused, and better connected to other information resources.

Type
Research Article
Information
Art Libraries Journal , Volume 48 , Issue 3 , July 2023 , pp. 74 - 79
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © The Author(s), 2023. Published by Cambridge University Press on behalf of ARLIS

The medium of comics is a dynamic art form that has not only evolved with technology,Footnote 1 but also matured alongside the reader. Once relegated to dime store spinner racks, the modern comic book evolved as young readers moved on from classic superhero stories to mature tales that often placed their favorite characters within new dilemmas and moral contexts.Footnote 2 As the stories evolved, so did the format, and the floppy, newsprint comic book transitioned from drugstore to bookstore as the hardcover graphic novel.Footnote 3 Today comics are celebrated within popular culture, easily found on library shelves, and the focus of critical analysis and scholarship across a variety of disciplines.Footnote 4 The typical comic story has also evolved beyond traditional tales of action and adventure to incorporate increasingly complex narratives, like the illustration of historical events (e.g. Maus by Art Spiegelman, Kent state by Derf Backderf) and personal memoirs, including stories of mental health and illness (e.g. Marbles by Ellen Forney, RX by Rachel Lindsay). These stories and events are reframed and made accessible through the graphic medium of comics which distills often complex, challenging topics and histories to a simplified cartoon world of juxtaposed images and text, in which the reader mentally participates in the flow of the narrative, utilizing their own knowledge and experience to bring closure to the space between pages and panels.Footnote 5

A current and relevant area of modern comic book and graphic novel storytelling is the intersection of comics and healthcare, or graphic medicine.Footnote 6 This genre is non-exclusiveFootnote 7 and includes any comic that communicates medical and healthcare information, or conveys stories of health, illness and recovery.Footnote 8 These works may not only help educate, but can also increase understanding and empathy between patients, caregivers and healthcare providers,Footnote 9 while enabling better insight into the patient experience.Footnote 10 With regards to mental health and illness, the visual medium of comics may provide a unique window into the mind of a person experiencing depression, anxiety, mania, etc.,Footnote 11 and may be destigmatizing when compared to other media such as television or movies, which often broadcast negative portrayals of mental illness that can influence the perception of caregivers.Footnote 12

While comic book and graphic novel data can be found in many bibliographic catalogs, databases and indexes, the metadata for works that can be categorized as graphic medicine often lacks references to healthcare-related knowledge organization structures (including taxonomy, controlled vocabularies, thesauri and ontologies) that would more precisely describe their contents. These distinct pages and panels may illustrate specific disorders, symptoms, side-effects, or other medical events and challenges — topics that often overlap and are shared with other illnesses, such as panic, anxiety and depression.

The implementation of a Linked Open Data (LOD) approach to describing graphic medicine may help enrich the description and discoverability of this distinct comics content, while also enhancing its potential to be remixed, reused and connected with other information resources. This article introduces a Linked Data (LD) model for describing the visual structure of comics content, and explores the application of that model in an abbreviated set of examples from States of mind, a graphic novel and personal memoir of bipolar disorder.

Comics community cataloging, indexing and other data projects

The comic book was not a publication format that received dutiful cataloging or preservation by knowledge institutions in its early years, and the responsibility of cataloging and indexing comics has traditionally been the activity of comics readership.Footnote 13 In the 1970s and 1980s, comics historian George Olshevsky and others worked to create indexes for major Marvel and DC Comics publications, providing the first detailed indexes of modern superhero stories from the major publishers.Footnote 14 In the 1990s this activity would move online with projects like the Grand Comics Database (GCD), an internet-based nonprofit and open database supported by a team of international volunteers.Footnote 15 Today the tradition of detailed hobbyist cataloging and indexing continues, and is joined by the efforts of libraries and universities that have made their datasets freely available, like the Comics as Data North America (CaDNA) project from the Michigan State University (MSU) Comics Art Library,Footnote 16 which includes the connection of that data to knowledge platforms like Wikidata.Footnote 17 Additionally, databases like eDBthequeFootnote 18 from the University of LaRochelle explore applying new technologies such as deep learning and natural language processing to the automated extraction, indexing and semantic annotation of digital comics content.Footnote 19 And cross-media projects like the Japanese Visual Media Graph (JVMG)Footnote 20 propose aggregating enthusiast data covering similar materials, including manga and visual novels, and making it more easily accessible to researchers — highlighting the significance of detailed, community created data in describing popular visual media.

Visual vocabulary of comics structure and content

The medium of comics communicates stories and ideas using sequential art, the juxtaposition of images, text and other pictorial elements on a page.Footnote 21 This common visual vocabulary of comics storytelling includes the use of panels to frame scenes within a story, balloons to convey speech, thought, or sound, and captions to incorporate narrative content. The negative space that forms around these panels is referred to as the gutter, which functions to not only visually separate panels, but also to help shape the perception of time and space within a sequence.Footnote 22 Within this empty space, the reader mentally participates in construction of the narrative by applying their own experience to bring closure between two sequences.Footnote 23 The concept of closure — perceiving the whole story through parts — is critical to reading and understanding comics. As Scott McCloud states in Understanding comics, ‘If visual iconography is the vocabulary of comics, closure is its grammar,’Footnote 24 or rather, more illustratively: comics sequences have ‘Closure for blood, gutters for veins…’Footnote 25

While the visual iconography of comics vocabulary can be reduced to common components, the precise identification of these elements is more difficult: a panel may have no border; a balloon may exist without a panel; the gutter may ‘bleed’ to the edge; art may ‘splash’ across multiple pages;Footnote 26 and so on. And while most comics are plotted with text intended to be read in a specific order, not all comics art has a specific sequence, and creators have little control over how the reader will interpret the page or where they might begin reading.Footnote 27

Comics metadata vocabulary

The visual vocabulary of comics can be transformed into digital data, or encoded, using several metadata schema: ComicsML is an XML-based markup language for online comics;Footnote 28 the Advanced Comic Book Format (ACBF) is a distribution and interchange format for digital comics;Footnote 29 the Comic Book Markup Language (CBML) is an XML vocabulary for encoding comics based on the Text Encoding Initiative Guidelines (TEI);Footnote 30 A Comics Ontology is an OWL ontology focused on narrative elements;Footnote 31 and the Manga Metadata Framework supports the description of digital manga.Footnote 32 Additionally, comics related terms were added to Schema.org in 2015 as part of the bib.schema.org extension. However, with the exception of Schema.org, these metadata vocabularies are either not intended to or do not appear to currently meet the technical requirements for publishing Linked Data (LD), such as providing dereferencable URIs under a persistent namespace, or various representations of a resource using content negotiation.

Comic Book Ontology and Linked Open Data

The Comic Book Ontology (CBO) is an OWL ontology and RDF metadata vocabulary for describing comic books and comic book collections.Footnote 33 It is open source, open licensed (CC BY 4.0) and publicly maintained on GitHub where it is open to feedback and contributions. CBO is not a replacement for existing metadata schema like ComicsML, CBML or Schema.org, rather as an LD vocabulary it can be used alongside other schema to create new records, or complement and enhance existing data through the process of semantic enrichment — a strategy that libraries, archives and museums have used to successfully enhance the discoverability and reuse of their data.Footnote 34

Linked Data (LD) refers to machine-readable structured data that is linked together through the use of standard protocols and common vocabularies.Footnote 35 LD utilizes an abstract data model called the Resource Description Framework (RDF) expressed as sentence-like ‘triples’ composed of a subject, predicate and object (e.g. Mary — subject — knows — predicate — Bob — subject). These RDF triples, or statements, compose the edges and nodes of a graph data structure, and when identifiers (e.g. URIs or web addresses) are used instead of text, machines (i.e. semantic web applications) can build connections between resources, discovering additional information through an interlinked, networked knowledge graph.

Linked Open Data (LOD) builds on the principles of LD by encouraging the utilization and publication of open licensed datasets and vocabularies,Footnote 36 or ontologies — knowledge organization structures that formally describe the classes and properties within a domain and the relationships between those concepts (including relationships to other ontology). When structured data is made freely available on the open web using non-proprietary, freely-licensed protocols and vocabularies, the effective knowledge graph for a given subject is free to expand and be queried semantically across open datasets, and across knowledge domains.

LD and RDF can be encoded using a variety of data formats (e.g. XML, JSON-LD, etc.), or embedded in HTML using microformats or RDFa. This article takes a ‘no-code’ approach, visualizing the examples presented as a graph diagram. However, all related code and technical examples are available in a companion GitHub repository: https://github.com/comicmeta/LOD-MentalHealth.

CBO Sequence data model

The CBO sequence model (cbo:Sequence) maps the vocabulary of comics structure and content to a set of classes: stories (cbo:Story), pages (cbo:Page), panels (cbo:Panel), captions (cbo:Caption), and balloons (cbo:Balloon) — distilling the visual language of comics to a simplified data model that can be mapped to RDF and aligned with other vocabulary, like Schema.org (schema:CreativeWork, schema:ComicStory). This updated model (as of v0.16) acknowledges both the literary and visual components of a complete comics sequence as it unfolds across pages, panels, and serialized stories. Additionally, the updated sequence model (see Fig. 1) attempts to conceptualize the grammar of comics by introducing a class for the gutter (cbo:Gutter). Similar to the visual element it represents, this class can also be used to group visual elements, providing a placeholder for describing otherwise invisible data related to closure (e.g. varying reader perspectives of a sequence).

Fig. 1. The Comic Book Ontology (CBO) Sequence model mapped to an excerpt from the story “Cut of the cards,” appearing in Dear lonely hearts (Comic Media, 1953), issue 2. Public domain. https://digitalcomicmuseum.com/preview/index.php?did=18196.

There are no strict constraints defined between these classes (i.e. there are no cardinality requirements or strict hierarchical rules) and the model requires minimal semantic and ontological commitment, acknowledging that there is often great visual variety in the expression and medium of comics (a term which may refer to publications, stories, artwork, etc.), and that comic book characters and stories often appear within or are adapted to other media, such as manga, movies and video-games (i.e. a comic can belong to multiple classes: a book, page of artwork, scene from a video-game, etc.).

A graphic medicine example

Comics have a long history in the application of sequential art for functions like storyboarding or illustrating instructional content.Footnote 37 Graphic medicine is not only an extension of this application, but also an expansion in how comics are analyzed and interpreted. In addition to the casual comics reader, there are at least three potentially differing reader perspectives when considering works of graphic medicine: patient, caregiver and healthcare provider.Footnote 38 These readers may interpret sequences quite differently — the knowledge they bring to the process of closure providing unique dimensions of experience and insight between panels.

The graphic novel States of mind by Emilie and Patrice Guillon is a personal memoir of bipolar disorder — a mood disorder that includes emotionally-high periods called mania, and low periods of depression.Footnote 39 This work is a unique example of graphic medicine, as both creators embody different reader perspectives: Emilie as patient, and father and co-creator Patrice as caregiver. The writers periodically break the narrative and provide a window into the often challenging and deeply personal creative process, as they pause to discuss and recall the events illustrated in the story, with occasionally differing recollections.

Example LOD graph using the CBO sequence data model

In the following example, Library of Congress Subject Headings (LCSH)Footnote 40 and BioPortal (a repository of biomedical ontologies)Footnote 41 were used for term discovery, with preference given to the Medical Dictionary for Regulatory Activities Terminology (MedDRA) and the National Cancer Institute Thesaurus (NCIT), both of which are open licensed, and/or freely available for non-commercial use. This article uses a shorthand when referring to these and other LOD resources that replaces the full URI (e.g. https://www.comicmeta.org/cbo) with a prefix (e.g. cbo).

While there are numerous sequences within States of mind relevant to mental health, there are several key examples LOD may enhance: a) illustration of medical topics like a specific condition or disease, such as bipolar disorder (lcsh:sh85080541); b) symptoms like stress (meddra:19942209) or panic attacks (meddra:10033664); c) medications and side-effects, like the antipsychotic Tercian (ncit:C79120) and loss of appetite (ncit:C113630); and d) medical events or experiences, like an involuntary commitment (meddra:10080523). The latter is a poignant example of varying reader perspectives — a patient may recall the loss of freedom or a sense of fear (meddra:10016275), while a caregiver might mentally fill the space between these panels with a memory or feeling of guilt (ncit:C95555).

Above is a graph diagram (see Fig. 2) of RDF statements from the preceding summary. This representative knowledge graph incorporates identifiers from the GCD (licensed under CC BY-SA 4.0), which includes an index for the work and entries for each sequence or chapter.Footnote 42 The CBO sequence model is applied to extend and enrich the index by adding nodes for pages and panels, which are then connected (using the schema:about property) to healthcare-related LOD resources. If these nodes were expanded, they would then incorporate additional statements into the graph, such as bipolar disorder and variants like manic depression from the LCSH entry, as well as Tercian and the synonym Cyamemazine from NCIT. Additionally, the diagram demonstrates use of the gutter concept in an effort to capture reader perspective, although future vocabulary development may uncover a more expressive or effective approach.

Fig. 2. A graph diagram visualizing select content elements from the graphic novel States of mind that illustrate topics relevant to mental health and illness. Nodes in blue and red represent data from external sources, including healthcare-related vocabulary and ontology.

Conclusion

This article presents a Linked Data (LD) model for describing comics content, and a method for connecting that content to healthcare-related Linked Open Data (LOD) resources. This approach may be helpful for enhancing the discoverability of graphic medicine by building upon the tradition and practice of comics indexing to link specific content with medical vocabulary and ontology. Additionally, the LOD approach presented is multilingual, respectful of copyright and attribution, and applicable for non-traditionally published work, including crowd-funded and self-published materials. However, data validation is required, and future study may explore incorporating objective review by domain experts.

While graphic medicine does not replace mental healthcare, enhancing and extending the description of this work may potentially help support the discovery and reuse of accessible and destigmatizing comics materials in advancing the broader conversation around mental health and wellness.

References

1. McCloud, Scott, “The Infinite Canvas: Digital Comics,” in Reinventing Comics (New York: HarperCollins, 2000), 200–41Google Scholar.

2. Duncan, Randy and Smith, Matthew J., “The History of Comic Books, Part II: The Maturation of the Medium,” in The Power of Comics: History Form and Culture (Bloomsbury Publishing, 2009), 50–82Google Scholar.

3. Duncan and Smith, “The Maturation of the Medium,” 70.

4. Williams, Ian, “Graphic Medicine: Comics as Medical Narrative,” Medical Humanities 38, no. 1 (June 2012): 21–22CrossRefGoogle ScholarPubMed, https://doi:10.1136/medhum-2011-01009 3.

5. Scott McCloud, Understanding Comics (New York: HarperCollins, 1994), 62–69.

6. MK Czerwiec, Ian Williams, Susan Merrill Squier, Michael J. Green, Kimberly R. Myers, and Scott T. Smith, Graphic Medicine Manifesto (Penn State Press, 2020), 1.

7. Matthew Noe, “Graphic Medicine & Canonization: Are We on a Worrisome Path?” in New England Graphic Medicine Conference (2022), https://doi.org/10.13028/8wvv-x247.

8. Williams, “Comics as Medical Narrative,” 21–27.

9. Alice Jaggers, Matthew Noe, and Ariel Pomputius, “Graphic Medicine in Your Library: Ideas and Strategies for Collecting Comics About Health Care,” in The Library's Guide to Graphic Novels, ed. John Ballestro (Chicago: ALA Editions, 2020), 167.

10. Michael J. Green and Kimberly R. Myers, “Graphic Medicine: Use of Comics in Medical Education and Patient Care,” BMJ 340, no. 7746 (2010): 576–77, https://doi.org/10.1136/bmj.c863.

11. Sweetha Saji and Sathyaraj Venkatesan, “Nobody Memoirs as Counter-Discourse: Bipolar Disorder and Its Metaphors,” in Metaphors of Mental Illness in Graphic Medicine (Routledge, 2021), 69–88, https://doi.org/10.4324/9781003214229-5.

12. Saji and Venkatesan, “Bipolar Disorder and Its Metaphors,” 87–88.

13. David S. Serchay, “Comic Book Collectors: The Serials Librarians of the Home,” Serials Review 24, no. 1 (1998): 57–70.

14. “In Memoriam: George Olshevsky,” Scoop, accessed March 15, 2023, https://scoop.previewsworld.com/Home/4/1/73/1012?ArticleID=257503.

15. “General FAQ,” Grand Comics Database, last modified September 12, 2022, https://docs.comics.org/wiki/General_FAQ.

16. Kate Topham, Julian Chambliss, Justin Wigard, and Nicole Huff, “The Marmaduke Problem: A Case Study of Comics As Linked Open (Meta)data,” KULA: Knowledge Creation, Dissemination, and Preservation Studies 6, no. 3 (2022): 1–8, https://doi.org/10.18357/kula.225.

17. Topham, Chambliss, Wigard and Huff, “The Marmaduke Problem,” 2–6.

18. See https://ebdtheque.univ-lr.fr/.

19. Nguyen, Nhu-Van, Rigaud, Christophe, and Burie, Jean-Christophe, “Digital Comics Image Indexing Based on Deep Learning,” Journal of Imaging 4, no. 7 (2018): 89CrossRefGoogle Scholar. https://doi.org/10.3390/jimaging4070089.

20. Magnus Pfeffer and Martin Roth, “Japanese Visual Media Graph: Providing Researchers with Data from Enthusiast Communities,” in International Conference on Dublin Core and Metadata Applications, (2019): 136–141, https://dcpapers.dublincore.org/pubs/article/view/4259.

21. McCloud, Understanding Comics, 5–9.

22. McCloud, Understanding Comics, 94–106.

23. McCloud, Understanding Comics, 60–63.

24. McCloud, Understanding Comics, 67.

25. McCloud, Understanding Comics, 73.

26. Eisner, Will, Comics and Sequential Art: Principles and Practices from the Legendary Cartoonist (W.W. Norton & Company, 2008), 39–101Google Scholar.

27. Eisner, Comics and Sequential Art, 40–42.

28. Jason McIntosh, “ComicsML: A Proposed Simple Markup Language for Online Comics,” last modified November 17, 2005, http://comicsml.jmac.org/about.html.

29. Robert Kubik, “Advanced Comic Book Format,” last accessed March 15, 2023, https://launchpad.net/acbf.

30. John A. Walsh, “Comic Book Markup Language: An Introduction and Rationale,” Digital Humanities Quarterly 6, no. 1 (2012), http://www.digitalhumanities.org/dhq/vol/6/1/000117/000117.html.

31. Paul Rissen, “A Comics Ontology — Explained,” last modified April 19, 2012, https://paulrissen.com/2012/04/19/a-comics-ontology-explained.

32. Morozumi, Ayako, Nomura, Satomi, Nagamori, Mitsuharu, and Sugimoto, Shigeo, “Metadata Framework for Manga: A Multi-Paradigm Metadata Description Framework for Digital Comics,” in International Conference on Dublin Core and Metadata Applications, (2009): 61–70Google Scholar.

33. Sean Petiya, “Comic Book Ontology,” last modified March 15, 2023, https://comicmeta.org/cbo/.

34. Marcia Lei Zeng, “Semantic Enrichment for Enhancing LAM Data and Supporting Digital Humanities. Review Article,” El Profesional de la Información 28, no. 1 (2019), https://doi.org/10.3145/epi.2019.ene.03.

35. Tim Berners-Lee, “Linked Data,” last modified June 6, 2009, https://www.w3.org/DesignIssues/LinkedData.html.

36. Berners-Lee, “Linked Data.”

37. Eisner, Comics and Sequential Art, 147–157

38. Jaggers, Noe, and Pomputius, “Graphic Medicine in Your Library,”165–69.

39. “Bipolar Disorder,” NIMH, last reviewed February, 2023, https://www.nimh.nih.gov/health/topics/bipolar-disorder.

40. “Library of Congress Subject Headings,” LOC, last modified February 14, 2023, https://id.loc.gov/authorities/subjects.html.

41. Patricia L. Whetzel, Natalya F. Noy, Nigam H. Shah, Paul R. Alexander, Csongor Nyulas, Tania Tudorache, and Mark A. Musen, “BioPortal: Enhanced Functionality via New Web services from the National Center for Biomedical Ontology to Access and Use Ontologies in Software Applications,” Nucleic Acids Research 39, no. suppl_2 (2011): W541-W545, https://doi:10.1093/nar/gkr469.

42. The author originally added these sequences to the GCD index for this book.

Figure 0

Fig. 1. The Comic Book Ontology (CBO) Sequence model mapped to an excerpt from the story “Cut of the cards,” appearing in Dear lonely hearts (Comic Media, 1953), issue 2. Public domain. https://digitalcomicmuseum.com/preview/index.php?did=18196.

Figure 1

Fig. 2. A graph diagram visualizing select content elements from the graphic novel States of mind that illustrate topics relevant to mental health and illness. Nodes in blue and red represent data from external sources, including healthcare-related vocabulary and ontology.