7.1 Introduction
Back in 2015, the international community committed, within the framework of the United Nations Sustainable Development Goals, to ensure healthy lives and promote well-being for all, and at all ages, by 2030 (United Nations, 2015). To measure progress toward this ambitious but necessary goal, we can assess the ease with which individuals can access healthcare. Some decades ago, the primary sources that people relied upon to seek medical services were mainly health professionals (e.g., doctors or pharmacists), or family and friends. At present, however, the web has become a ubiquitous health knowledge center that allows us to seek needed information quickly and privately. Now that people increasingly depend on the Internet to make informed health decisions – for example, to diagnose or treat given conditions, or to obtain lifesaving information in a crisis, as during the COVID-19 pandemic – it is essential to ensure access to web health content for all potential users.
The need becomes even more acute for people with disabilities (PwD), as they are more likely than others to use the Internet for health-related activities (Reference ScanlanScanlan, 2021). Guaranteeing effective access to healthcare for PwD is a key action point under the recently published Strategy for the Rights of Persons with Disabilities 2021–2030 (European Union, 2021). However, evidence suggests that PwD report unmet needs for medical-related services four times more often than persons without disabilities (ibid.), and that their satisfaction rates with their medical care in general are lower than those of their non-disabled peers (Reference Gibson and O’ConnorGibson and O’Connor, 2010). In times of crisis, these challenges can be exacerbated and make this population group more vulnerable, particularly during the response and recovery stages (Reference Rodríguez Vázquez, O’Brien and FedericiRodríguez Vázquez, 2023). In fact, after the COVID-19 outbreak, the United Nations (2020) acknowledged that PwD are at greater risk of discrimination in accessing healthcare and lifesaving procedures in emergency contexts. And yet, despite all this clear evidence of need, prior work indicates that key health websites, such as that of the World Health Organization, have failed to serve the information needs of all citizens during the health crisis (Reference Fernández-Díaz, Iglesias-Sánchez and Jambrino-MaldonadoFernández-Díaz et al., 2020). If health websites are to achieve their ultimate informative goal and serve the needs of diverse communities – whether functionally, culturally, or linguistically (Reference Rodríguez Vázquez, Torres-del-Rey, Federici and O’BrienRodríguez Vázquez and Torres-del-Rey, 2020) – their content must also be available in a language they can understand. The suggestion has been made that provision of multilingual health counseling and information services could efficiently (i) reduce short-term costs to health insurers of searches for health information and (ii) improve patient empowerment (Reference Schmidt, Wild and SchreyöggSchmidt et al., 2021). However, we contend that this provision should also be seen as a form of accessibility in itself. We also believe that health websites offering information in several languages – especially those from concerned institutions in multilingual countries – must ensure that an acceptable and comparable level of accessibility is achieved across all language versions, per the Web Content Accessibility Guidelines (WCAG) 2.1 (Reference Kirkpatrick, Connor, Campbell and CooperKirkpatrick et al., 2018).
In the present chapter, we will first review the existing literature on the topic of accessibility and localization of health websites (Section 7.2). Then we will present the methodology and the results of a study aiming to evaluate a selection of official multilingual health websites for accessibility compliance, with a particular focus on two language-oriented WCAG success criteria: 2.4.2, Page Titled and 3.1.1, Language of Page (Sections 7.3 and 7.4). The chapter will conclude with a discussion of the study’s findings, of the challenges that can emerge from a localization-oriented accessibility evaluation, and of several ideas for continuing this line of research (Section 7.5).
7.2 Related Work
7.2.1 Accessibility of Health Websites
Health websites have been the focus of several studies investigating various aspects of information accessibility, including many issues surrounding availability, accessibility, and affordability of information.Footnote 1 For instance, some researchers have placed emphasis on studying the reliability and trustworthiness of health content online (Reference Hamzehei, Ansari, Rahmatizadeh and Valizadeh-HaghiHamzehei et al., 2018), while others have assessed the availability and quality of information about specific conditions, such as cancer (Reference Lawrentschuk, Sasges and TasevskiLawrentschuk et al., 2012), benign prostatic hyperplasia (Reference Chen, Manecksha and AbouassalyChen et al., 2014), or female urinary incontinence (Reference Saraswat, Abouassaly, Dwyer, Bolton and LawrentschukSaraswat et al., 2016), among others.
Considerable literature has also been published on the compliance of health websites with the accessibility guidelines created by the World Wide Web Consortium (W3C), ranging from the first WCAG 1.0 version (Reference Chisholm, Vanderheiden and JacobsChisholm et al., 1999) to the most recent WCAG 2.1 (Reference Kirkpatrick, Connor, Campbell and CooperKirkpatrick et al., 2018).Footnote 2 Most studies we identified involved the classic web accessibility (WA) audits and error reports, in the form of proactive or reactive evaluation studies (Reference VigoVigo 2009). The former usually support an iterative accessible development process by helping content producers identify lists of problems to fix and are therefore formative (Reference Brajnik, Hartmann, Zhou and KirchbergBrajnik 2008). This was the approach adopted by Reference Acosta-Vargas, Rybarczyk and PérezAcosta-Vargas et al. (2018) for a web platform that enables home support as a patient recovers after an arthroplasty. By following the Website Accessibility Conformance Evaluation Methodology (WCAG-EM) 1.0 (Velleman and Abou-Zahra, 2014) and including both automatic and manual testing, the authors identified key issues that could be corrected at a later stage, thus improving the platform’s overall accessibility.
Most of the other retrieved studies, however, opted for a reactive evaluation (Reference Brajnik, Hartmann, Zhou and KirchbergBrajnik, 2008): the tests were carried out after the websites had been released, simply to assess or validate their accessibility levels. Not surprisingly, results across studies were similar, regardless of the method used – whether automatic testing was applied alone (the most popular choice) or in combination with human evaluation. When health websites in Italy (Reference Mancini, Zedda and BarbaroMancini et al., 2005) and Canada (Reference O’GradyO’Grady, 2005) were automatically checked for compliance against WCAG 1.0, over 60 percent of the sampled sites failed to comply with the minimum accessibility requirements. Similarly, Reference Zeng and ParmantoZeng and Parmanto (2004) discovered that, although government and education health websites obtained better accessibility scores than the other web portals in their corpus of 108 worldwide sites (corporate, e-commerce, and community health websites), none were in fact fully accessible.
Low levels of accessibility compliance were also found in more recent studies that took the WCAG 2.1 as a baseline. These include, to name but a few, studies by Reference Rahmatizadeh and Valizadeh-HaghiRahmatizadeh and Valizadeh-Haghi (2018) on medical university websites in Iran; Reference Acosta-Vargas, Rybarczyk and PérezAcosta-Vargas et al. (2018) in a corpus of twenty-two hospital websites chosen by following standard Webometrics rating criteria; and Reference AlajarmehAlajarmeh (2021), who assessed public health websites of the top twenty-five countries affected by the COVID-19 pandemic in 2020. Studies focusing on the needs of a particular group of users yielded similar conclusions. Of the 139 websites containing medical information addressing blind and visually impaired (BVI) laymen or patients explored by Reference Lüchtenberg, Kuhli-Hattenbach, Sinangin, Ohrloff and SchalnusLüchtenberg et al. (2008), only 18 percent (15 sites) achieved level A or AA. Similarly, findings from a study conducted by Reference YiYi (2020) with 25 BVI on the accessibility of ten government and public agency health websites in Korea revealed that all of them presented barriers for the relevant participants. Lastly and notably, some scholars have also looked into other accessibility-related variables that could enhance the end-user experience on medical information sites. For instance, in a usability experiment with American Sign Language (ASL) users, Reference Kushalnagar, Naturale and PaludnevicieneKushalnagar et al. (2015) showed that making health websites accessible in ASL is insufficient, as the sites must also be user-friendly and easy to navigate. Finally, another interesting study by Reference YoungbloodYoungblood (2020) on the mobile readiness of twenty-five of the top health information website homepages demonstrated that, while the overwhelming majority of the sites were at least partially mobile-ready, paradoxically enough, many of the sites violated critical accessibility guidelines.
Our contribution to the current literature is two-fold. First, by adopting a localization approach to WA evaluation, we make what we believe is the first attempt to systematically check accessibility features across two different language versions in multilingual health-related websites. Although most of the studies mentioned in the present section included in their test samples websites with content in several languages (e.g., Reference Berland, Elliott and MoralesBerland et al. (2001), Reference Acosta-Vargas, Rybarczyk and PérezAcosta-Vargas et al. (2018) and Reference AlajarmehAlajarmeh (2021), to name but a few), all the accessibility assessments they conducted were language-independent. Second, although prior work has investigated the compliance of health websites with success criterion (SC) 2.4.2 and SC 3.1.1, checks were mostly limited to verifying that the lang attribute was present and whether or not the title attribute in the head element was empty. As we will explain in Section 7.3, our study will analyze both success criteria in context, thus reducing the risk of bias in the results due to false positives.
7.2.2 Localization of Health Websites
In Reference Jiménez-Crespo, Valdeón and Vidal ClaramonteJiménez-Crespo’s terms (2019: 354), industry’s and society’s prototypical understanding of web localization could be summarized as follows: such localization concerns the translation of interactive hypertexts, but entails a specific set of technological and management processes, such as web content management systems and other web-specific technologies, which are not shared with other translation practices. In addition, web localization requires human intervention; that is, “an instant translation of [web content] using any [Machine Translation] widget in websites without any post-editing or human intervention might not be considered in the industry as an exemplar of the prototype” (Reference Jiménez-Crespo, Valdeón and Vidal ClaramonteJiménez-Crespo, 2019: 355). Most importantly, “web localization operates exclusively on digital web genres” (Reference Jiménez-Crespo, Valdeón and Vidal ClaramonteJiménez-Crespo, 2019: 355) – that is, genres used only online.Footnote 3
Considering Reference Jiménez-CrespoJiménez-Crespo’s (2013: 95–100) proposal concerning the web genre category, we could define health websites as composing an informational web genre, aiming to both provide information (its expositive function) and to modify user behavior (its exhortative function). For example, a health website may contain descriptive information about a given condition but also recommendations related to its treatment or general health habits. For our work, we understand that these portals can be institutional, nonprofit, or community association websites, targeting either health professionals or lay users.
Existing research recognizes the “medical information website”Footnote 4 as the genre with the highest volume of translation around the word (Reference Jiménez-Crespo and Tercedor SánchezJiménez-Crespo and Tercedor Sánchez 2017: 412), considering it as a modern version of the classic patient health information leaflet. Surprisingly, nonetheless, it has received scant attention in the localization literature. The very few studies related to health website localization that do exist are based on a web corpus of sites from the United States (US) – an intriguing finding in itself, given that the US is not officially a multilingual country. Perhaps one reason for this decision is precisely the vulnerability of speakers of (official or non-official) minority languages with respect to information access. As Reference Piller and BoomgaardenPiller (2020: 14) rightfully puts it:
For too long, state approaches to speakers of minority languages – whether indigenous or migrant – have ranged from benign neglect to forced assimilation. In order to gain access to the state and its institutions – education, health, welfare or the law – everyone was expected to speak the language of the state – English in the US, French in France, Mandarin in China, and so on. As a result of such monolingual approaches, Spanish speakers in the US, Arabic speakers in France, or dialect speakers in China have worse education, employment and health outcomes than their compatriots speaking the state language.
For the purposes of the present chapter, we will refer to three studies in particular. The first focused on the textual and structural analysis of nonprofit US websites across language versions (English and Spanish), including websites of healthcare organizations, in order to study content loss (Reference Jiménez-CrespoJiménez-Crespo, 2012). According to Reference Jiménez-CrespoJiménez-Crespo (2012), the United States offers more non-profits with a wider range of social services than any other Western nation. Upon examining the corpus, the author reported, as a key finding, that the overall probability that sections of the source website would not be localized were 44.18 percent, implying that, on average, almost half the sections of any source website will not be localized into Spanish. In a subsequent study, Reference Jiménez-CrespoJiménez-Crespo (2017) used the Translational Web Corpus of Medical Spanish (TWCoMS) – containing medical information websites addressing general audiences in the US and a comparable section of websites for Mexico and Spain – for a user evaluation study with twenty-five Spanish speakers living in the State of New Jersey. The goal was to assess users’ preferences regarding reformulations and explicitations. His hypothesis was that medical texts translated from English into Spanish would be easier for end users to understand than their non-translated counterparts and thus preferred. However, the data collected in the human evaluation suggested otherwise: bilingual Spanish/English speakers living in the US preferred the most frequent reformulations explicitations in non-translated texts (Reference Jiménez-CrespoJiménez-Crespo, 2017). Their initial assumption was based on the results of a parallel study with the same corpus, in which US medical websites translated into Spanish displayed lower register and lexical specialization levels, with more frequent reformulation strategies than similar non-translated ones (Reference Jiménez-Crespo and Tercedor SánchezJiménez-Crespo and Tercedor Sánchez, 2017).
Notwithstanding the relevance of the aforementioned studies and their notable contribution to the web localization field, their focus was not on the needs of PwD. Similarly, accessibility compliance was not the main variable investigated, in contrast to our study and to the research that will be reviewed in the following section.
7.2.3 Multilingual Web Accessibility Studies
Truth be told, despite the inherent multilingual nature of the web, there is a general paucity of scientific literature specifically relating to the accessibility of localized websites. In the last ten years, continuous efforts have been devoted to the advancement of research on the knowledge and resources required to create multilingual web content for all.
Concretely, scholars have explored, on one hand, various process-related aspects, for example, ways in which existing technology can support the multiple stakeholders involved in the development of multilingual websites to render the sites accessible. These include, among others, studies involving the evaluation of how WA evaluation tools deal with language-related issues (Reference Rodríguez VázquezRodríguez Vázquez, 2016a) and mobile-related aspects (Reference Morado Vázquez, Torres-del-Rey, O’Brien and FedericiMorado Vázquez and Torres-del-Rey, 2023). Additional studies have also examined whether accessibility can be supported through the use computer-assisted translation (CAT) tools (Reference Pacati, Rodríguez Vázquez, Jekat, Puhl, Carrer and LintnerPacati and Rodríguez Vázquez, 2021), localization data exchange standards (Reference Torres-del-Rey and Morado VázquezTorres-del-Rey and Morado Vázquez, 2019), and controlled language checkers (Reference Rodríguez VázquezRodríguez Vázquez, 2015a) – and if so, how. On the other hand, product-oriented studies have investigated accessibility features in localized websites corresponding to specific WCAG guidelines (Reference Kirkpatrick, Connor, Campbell and CooperKirkpatrick et al., 2018). Examples include the work of Reference Rodríguez VázquezRodríguez Vázquez (2016b) on the appropriateness of text alternatives for images (Guideline 1.1 Text Alternatives), or the observation exercise conducted by Reference Rodríguez Vázquez, Torres-del-Rey, Morado Vázquez and Sánchez RamosRodríguez Vázquez et al. (2022) on the use of easy language in multilingual websites, which could be understood as a good practice to meet Guideline 3.1 Readable.
The web elements (the language and title of the page) that are central to the two WCAG success criteria, and which we will analyze in depth in our study (see Section 7.3), have been considered from various perspectives in prior work related to localization. For instance, in their proposal of a heuristic evaluation methodology to assess multilingual websites, Reference Andreu-Vall and MarcosAndreu-Valls and Marcos (2012) included the language of the page as an internationalization feature, while the verification of the page’s title was recommended for SEO purposes, and thus not necessarily focusing upon these elements’ added value for accessibility. Similarly, Reference Jiménez-CrespoJiménez-Crespo (2008) carried out a lexical analysis of the web page titles in an English-Spanish corpus of US corporate websites. He found that proper names were used in 25 percent of the cases. Interestingly enough, the terminology study also revealed the use of meaningless words, corresponding to website domain suffixes, such as “com” in the English subcorpus and “es” in the Spanish subcorpus. Although examination of the lang attribute was not among the goals of the study, it was reported as part of the corpus metadata: the author noted that around 30 percent of the pages in the localized web subcorpus had said attribute, only 25 percent of which actually had an appropriate language value.
The most recent multilingual WA studies focused on a varied range of web genres. Reference CasalegnoCasalegno (2018) investigated the usability of two partially localized university websites for BVI people. During user testing, participants reported issues related to the inability of their screen readers (programs that automatically read aloud content visually represented on the screen) to correctly read the content of the pages, a problem stemming from the incorrect implementation of the lang attribute. This finding agreed with those reported by Reference Rodríguez VázquezRodríguez Vázquez (2015b) after a series of interviews with members of the BVI community about the challenges they usually faced when browsing multilingual websites. The two success criteria of our interest were also analyzed by Reference MinacapilliMinacapilli (2018) and Reference PontusPontus (2019) in multilingual airline and museum websites, respectively, within the framework of larger accessibility evaluation studies on compliance to language-related accessibility best practices. After combining automatic and human evaluation by a single inspector, they both concluded that the localized versions were less accessible than the original ones, both at a general level and in relation to compliance with SC 2.4.2 and SC 3.1.1.
Our ultimate goal is to explore whether these asymmetrical situations are found in health websites, and to recommend possible ameliorations that can promote universal access to multilingual health information for all. The study to be presented in the following sections differs from and complements prior work in that: (i) it proposes a localization-oriented accessibility evaluation of the two aforementioned success criteria in a new web genre: health websites; (ii) it includes a more in-depth and in-context analysis of both success criteria in two different language versions; and (iii) it involves a manual inspection step, conducted by more than one accessibility expert.
7.3 Methodology
As explained, our work aims to address the accessibility of health websites from a multilingual perspective. Concretely, we sought to answer the following main research question (RQ):
RQ1. Do multilingual websites provide the same level of access to health information in all their language versions?
For that purpose, we studied the home pages of a set of multilingual websites in English and Spanish providing health-related information (see Section 7.3.1) and automatically evaluated their accessibility (see Section 7.3.2). Taking into account the conclusions drawn in prior work with regard to the accessibility of the multilingual web, our main hypothesis was that, overall, the original web pages would feature a higher level of accessibility than their corresponding localized versions.
In addition, we aimed to explore the following secondary research questions:
RQ 1.1. Is the language of the original and localized web pages defined in such a way that it can be programmatically determined?
RQ 1.2. Are the titles of the original web pages more accessible than the localized ones?
To answer these questions, we examined two specific language-related accessibility features in our multilingual web sample: the Title of the page and the Language of the page, which respectively correspond to the success criteria 3.1.1 and 2.4.2 of the WCAG 2.1 (Reference Kirkpatrick, Connor, Campbell and CooperKirkpatrick et al., 2018). Both criteria were analyzed through both automatic evaluation and manual inspection.
7.3.1 Data Selection
The multilingual websites included in our study were initially selected according to two main criteria: a) they should be sites that provide trustworthy health-related information, and b) they should be available in both English and Spanish.Footnote 5 Taking this into account, we decided to consult MedlinePlus, an online service of the US National Library of Medicine (NLM), whose mission “[…] is to present high-quality, relevant health and wellness information that is trusted and easy to understand, in both English and Spanish” (National Library of Medicine, 2022a). Considered the most visited health website in the world (Reference Acosta-Vargas and Acosta-VargasAcosta-Vargas and Acosta-Vargas, 2021), it contains a directory of health-related organizations “[…] whose materials appear on MedlinePlus health topic pages” (National Library of Medicine, 2022b). This particular directory proved valuable and convenient in the context of our research, as it listed 705 organizations at the time of data collection (May 2021); yet many of those organizations’ websites could not be integrated into our bilingual study because they were available only in English.
A third selection criterion was applied in relation to the level of localization of the sites in our sample: the localized version should include at least the home page and one of the main menus in the target language (Spanish in our case). Hence, we discarded all websites with a localization level ranging from 0 to 2, per the classification defined by Reference Jiménez-CrespoJiménez-Crespo (2013: 35–36), namely: a) websites that provide only isolated documents in Spanish (e.g., a PDF document about a specific event or topic); b) websites that include only a list of resources in Spanish; c) websites featuring only a short text or single page localized into Spanish, with all navigation menus in English; and d) websites localized through a third-party MT service without human post-editing (e.g. through a Google Translate plugin).
Considering these criteria, our final web sample consisted of the home pages of seventy-four websites available in English and Spanish (see Annex A). As acknowledged in prior work (Reference Acosta-Vargas, Rybarczyk and PérezAcosta-Vargas et al., 2018; Reference AlajarmehAlajarmeh, 2021), home pages are decisive for user experience, as they are the main entry point and, should problems arise, access to other pages within the same website could be compromised. Most of the sites (92 percent) were from US organizationsFootnote 6 and only six (8 percent) were from international ones. We hypothesized that all the websites were originally created in English (either because it is the main official language in the US or as a preferred ‘lingua franca’) and later localized into Spanish; this assumption was also supported in that most of the Spanish versions were shorter than the English websites (indicating partial localization). The web sample was stored locally and analyzed between May and September 2021.
7.3.2 Testing Methods
Automatic testing is a popular accessibility evaluation method, as automated tools provide a quick and low-cost mechanism for gathering accessibility information. For our purposes, we chose Google Lighthouse,Footnote 7 an open-source web development tool that includes a specific accessibility audit module. We ran it in Chrome DevTools and assessed the 148 home pages of our web sample: seventy-four pages in English and seventy-four in Spanish. We decided to use this automated tool in our study because it fulfills the transparency principles described in Reference O’GradyParvin et al (2021: 2):
a) It is clear which accessibility aspects are examined – see Reference Gibson and O’ConnorGoogle Developers (2019a). Among these, we find the aspects included in our study: the title and the language of the page.
b) The errors detected in the accessibility audits are properly categorized and presented. After conducting the accessibility analysis, the tool produces a clear report that can be downloaded in several formats for later reuse (Google Developers 2021). For each one of the accessibility audits performed by the tool, there are three possible values: not applicable, passed, or failed.Footnote 8 This report also provides an overall accessibility score using a 100-point scale. We exploited both results in our study. Similarly, the tool lists a series of items that should be implemented manually because they cannot be tested automatically – for example, checking that the page has a logical tab order (Google Developers, 2022a).
c) It provides clear information on the procedure for calculating a weighted average of all the accessibility audits (Google Developers, 2019a) to derive the overall score. That is, the weight of the audits depends on their accessibility impact (ibid.).
Nevertheless, automatic evaluation tools entail limitations, and it is usually recommended that their use as a single testing method be avoided: they should be combined with human evaluation whenever possible (Reference Abou-Zahra, Harper and YesiladaAbou-Zahra, 2008; Reference Brajnik, Hartmann, Zhou and KirchbergBrajnik, 2008). Human evaluation can be carried out by accessibility experts or with the help of end users, who can be requested to assess the website(s) according to a given scenario. In the current study, we compared the results obtained through Google Lighthouse with those from the manual inspection conducted by the two researchers.
For the human evaluation of the two accessibility features selected, we extracted the language values and the titles of the 148 pages of our web sample and included them in an evaluation template created ad hoc for the purposes of our study. For research reliability purposes, once the first evaluator concluded the manual inspection of all pages in the web sample, the second reviewed all the error annotations. Minor discrepancies were then discussed and resolved.
7.3.3 Accessibility Features Studied
As mentioned, we checked our web sample against two success criteria of the WCAG 2.1 (Reference Kirkpatrick, Connor, Campbell and CooperKirkpatrick et al., 2018): SC 3.1.1, Language of page, under the “Understandable” principle; and SC 2.4.2, Page Titled, under the “Operable” principle. Both success criteria are classified as level A, which is the minimum level of conformance defined in the WCAG 2.1 guidelines (Reference Kirkpatrick, Connor, Campbell and CooperKirkpatrick et al., 2018: sec. 5.2.1). The WCAG 2.1 stablishes three levels of conformance: A, AA and AAA, and current national and international accessibility regulations and normsFootnote 9 tend to recommend that websites meet the success criteria classified as level A and AA. In the sections that follow, these two aspects will be explained in detail.
7.3.3.1 Language of the Page
In the WCAG 2.1 (Reference Kirkpatrick, Connor, Campbell and CooperKirkpatrick et al., 2018), success criterion 3.1.1 (Language of Page) indicates that the “human language of each Web page can be programmatically determined.” This aspect can help assistive technology to programmatically prepare the content to suit the user’s needs: for example, the screen reader could automatically identify that the text is written in a specific language and pronounce it accordingly. In HTML (HyperText Markup Language), this information can be indicated by including the lang attribute in the <html> root element. There is a standardized list of language tags that is defined in the IETF’s BCP 47 standard.Footnote 10 For example: <html lang=”en”> indicates that the page is written in English, as “en” is the official value for English. It is also possible to include other subtag values, e.g., to indicate the region. For instance, <html lang=”en-US”> refers to English from the United States. However, as Reference IshidaIshida (2014) recommends, the golden rule is to keep the value as short as possible whenever possible.
In our analysis, firstly, we followed the test rules defined in (Reference Campbell, Cooper and KirkpatrickCampbell et al., 2022: sec. Understanding Success Criterion 3.1.1: Language of Page) and, through Google Lighthouse, we checked:
1. If the web pages included the lang attribute in the <html> element.
2. If the value of that attribute followed the IETF’s BCP 47 standard (Reference IshidaIshida, 2016).Footnote 11
Secondly, we manually verified whether the value of that attribute matched the language of the page’s content.Footnote 12 Finally, note that in HTML it is also possible to programmatically indicate that a specific section in a web page is written in a natural language different from the page’s main language. The use of the attribute lang should be used for this purpose in the corresponding HTML element(s). This aspect, which is covered in the SC 3.1.2 of the WCAG 2.1 entitled “language of parts” (Reference Kirkpatrick, Connor, Campbell and CooperKirkpatrick et al., 2018), was not examined in our analysis, but it would be interesting to cover it in future studies.
7.3.3.2 Title of the Page
In a web page’s HTML document, the title is located in the <title> element within the <head>. Usually it is visually represented at the top tab of a web browser. Under the SC 2.4.2 (Page Titled), the rule to be verified is that “Web pages have titles that describe topic or purpose” (Reference Kirkpatrick, Connor, Campbell and CooperKirkpatrick et al., 2018). The title of the page is especially important for users of screen readers – again, programs that read aloud content visually represented on the screen. As explained in Deque Systems (2022), the title of the page will be the first element that these users will hear when visiting a page, and if this information is not descriptive and unique, they will have to explore the page to determine its content and purpose. Similarly, as Reference BrajnikBrajnik (2009) suggests, “if the title provides no information or does not change when pages are changed, it gives the wrong hint to the user who might not understand that the page has changed at all.” The same applies if the page title is irrelevant or incorrectly translated.
The Web Accessibility Initiative (WAI) also provides a particular technique (G88) to help developers implement this criterion correctly.Footnote 13 In this technique, they prescribe the following rules to write descriptive page titles:
The title of each Web page should:
– Identify the subject of the Web page
– Make sense when read out of context, for example by a screen reader or in a site map or list of search results
– Be short
It may also be helpful for the title to
– Identify the site or other resource to which the Web page belongs
– Be unique within the site or other resource to which the Web page belongs”
For our analysis, taking these recommendations into account, as well as those from other key stakeholders in the field of accessibility (Reference Berners-LeeBerners-Lee, 1992; Reference Kirkpatrick, Connor, Campbell and CooperKirkpatrick et al., 2018; Reference White, Abou-Zahra and Lawton HenryWhite et al., 2020; Google Developers, 2022b; WHATWG, 2022: sec.4.2.2) and considering the researchers’ expertise on the subject, we decided to manually examine the titles of our web sample against the following compliance criteria:
1. The title is short, i.e., it does not contain more than sixty-four characters.Footnote 14
2. The title identifies the subject of the web page.
3. The title makes sense when read out of context.
4. The title does not include repetitions.Footnote 15
5. The title does not include abbreviations without the expanded form.
6. The title does not include URL addresses.
When analyzing the localized versions, we observed that this particular subsample had its own language-related issues, so we decided to add three new criteria to our initial list. Hence, we additionally assessed the titles in the Spanish web pages against the following criteria:
7. The localized title differs from the original title.Footnote 16
8. The text is in the target language.Footnote 17
9. The title is not composed of text in both the original and the target languages.
Google Lighthouse can determine only if the <title> element is present or not (Google Developers, 2019b). Consequently, the nine criteria described above were assessed only through manual inspection.
As can be inferred from the list, some of these criteria depend heavily on human judgment and can be examined only in context. For instance, it is not possible to determine if the title accurately describes the subject of the page without reading the content of the page (criteria 2 and 3). Similarly, criterion 7 requires a comparison between the title of two different pages, while automating the verification of criteria 8 and 9 would imply the use of specialized natural language processing (NLP) tools. Other criteria can be subjectively analyzed on the isolated titles themselves; for example, to determine if an abbreviation was used without its expanded form (criterion 5). Certain criteria can even be automatically measured, such as determination of the title length, which we accomplished by automatically calculating the total number of characters in our evaluation template (criterion 1).Footnote 18
It is worth highlighting here that several authors (Reference White, Abou-Zahra and Lawton HenryWhite et al., 2020; Accessibility Guidelines Working Group, 2022: sec.G88; Google Developers, 2022b; Mozilla and individual contributors, 2022) also recommend verifying that the titles of the web pages are unique within their websites. Nevertheless, we did not include this recommendation in the compliance criteria of our study because we focused on the home pages, and not on complete websites. In addition, we decided not to examine other recommendations and best practices which have SEO implications and/or that are more related to style preferences (e.g., “Brand your titles concisely” (Google Developers, 2022b)). Nonetheless, it would be pertinent to observe and assess them in future research.
7.4 Results
In order to answer our main research question, we collected and analyzed the overall accessibility scores of all the pages in our web sample, as provided by Google Lighthouse. These initial results indicated that the accessibility score of the 148 home pages thus analyzed was, on average, goodFootnote 19 (x̄ = 90, sd = 8.9). We did not find noteworthy differences between the results of the English (EN) home pages (x̄ = 90.1, sd = 8.6) and the Spanish (ES) ones (x̄ = 90, sd = 9.3). Therefore, if we were just to consider the results of the general automatic evaluation, we could conclude that the localized subsample (ES) seems to be as accessible as the original one (EN), thus tentatively disconfirming our initial hypothesis.
In a subsequent stage, we looked at the accessibility audits that were not met according to Google Lighthouse in each page, considered individually. Table 7.1 provides an overview of the errors detected by the tool, sorted from the most common to the least. The most recurrent errors were the following: “insufficient contrast ratio between the background and foreground colors,” which was present in 84 (57 percent) of the pages analyzed; “links did not have a discernible name,” present in 76 (51 percent) pages; “heading elements were not in a sequentially-descending order,” present in 55 (37 percent) pages; and “images elements did not have the alt attribute,” present in 40 (27 percent) pages. As we can see from the data in Table 7.1, according to Google Lighthouse’s results, there are only minor differences between the EN and ES subsamples. A full discussion of these unmet criteria is beyond the scope of this study, but would certainly merit further consideration in future research.
Table 7.1 Accessibility errors (total number and %) in the study web sample according to Google Lighthouse
| Type of error | EN | ES | Total |
|---|---|---|---|
| Background and foreground colors do not have a sufficient contrast ratio | 44 (59%) | 40 (54%) | 84 (57%) |
| Links do not have a discernible name | 39 (53%) | 37 (50%) | 76 (51%) |
| Heading elements are not in a sequentially-descending order | 29 (39%) | 26 (35%) | 55 (37%) |
| Image elements do not have [alt] attributes | 22 (30%) | 18 (24%) | 40 (27%) |
| Buttons do not have an accessible name | 12 (16%) | 14 (19%) | 26 (18%) |
| [user-scalable="no"] is used in the <meta name=”viewport”> element or the [maximum-scale] attribute is less than 5 | 8 (11%) | 8 (11%) | 16 (11%) |
| [id] attributes on active, focusable elements are not unique | 7 (9%) | 5 (7%) | 12 (8%) |
| Lists do not contain only <li> elements and script supporting elements (<script> and <template>) | 7 (9%) | 6 (8%) | 13 (9%) |
| [aria-hidden="true"] elements contain focusable descendants | 7 (9%) | 9 (12%) | 16 (11%) |
| ARIA IDs are not unique | 7 (9%) | 7 (9%) | 14 (9%) |
| <frame> or <iframe> elements do not have a title | 5 (7%) | 4 (5%) | 9 (6%) |
| ARIA input fields do not have accessible names | 5 (7%) | 3 (4%) | 8 (5%) |
| <html> element does not have a [lang] attribute | 4 (5%) | 5 (7%) | 9 (6%) |
| Some elements have a [tabindex] value greater than 0 | 3 (4%) | 4 (5%) | 7 (5%) |
| [role] values are not valid | 3 (4%) | 4 (5%) | 7 (5%) |
| [role]s do not have all required [aria-*] attributes | 3 (4%) | 3 (4%) | (6) 4% |
| <object> elements do not have [alt] text | 2 (3%) | 3 (4%) | (5) 3% |
| List items (<li>) are not contained within <ul> or <ol> parent elements | 2 (3%) | 2 (3%) | (4) 3% |
| [role]s are not contained by their required parent element | 2 (3%) | 1 (1%) | (3) 2% |
| [aria-*] attributes do not have valid values | 2 (3%) | 2 (3%) | (4) 3% |
| button, link, and menuitem elements do not have accessible names | 2 (3%) | 3 (4%) | (5) 3% |
| Form elements do not have associated labels | 1 (1%) | 1 (1%) | (2) 1% |
| Elements with an ARIA [role] that require children to contain a specific [role] are missing some or all of those required children. | 1 (1%) | 2 (3%) | (3) 2% |
| The page does not contain a heading, skip link, or landmark region | 0 (0%) | 1 (1%) | (1) 1% |
Note also that Google Lighthouse does not provide partially compliant results (Reference Gibson and O’ConnorGoogle Developers, 2019a), since it only provides a pass, fail, or non-applicable value for each of the forty-two accessibility audits that it performs. For example, if all of the image elements have alt attributes except for one, that page will entirely fail this specific accessibility audit. This approach, while effective, might not always reflect the impact of errors on the final user experience: for instance, omitting a text alternative for an image on the footer of the page might not have the same effect as omitting it for a content-rich image in the page’s main body. This rationale is consistent with the most recent scoring proposal proposed by the W3C in the draft of the new WCAG 3.0 now in progress, which explores various scoring mechanisms beyond binary true/false tests (Reference Spellman, Montgomery, Lauriat and CooperSpellman et al., 2021b: sec.5). Still, only empirical studies with end users would enable us to identify the real impact of partial violations of accessibility compliance.
7.4.1 Language of the Page
To study this accessibility feature in our web sample, we first analyzed the results from Google Lighthouse, and then compared them with those from the manual inspection. Overall, according to this automated tool, 139 out of the 148 (94 percent) pages defined the language of the page. The value of the lang attribute in those pages was also considered correct. In other words, as indicated in Table 7.1 (see the row with text in bold), nine pages were not compliant with this criterion according to Google Lighthouse, since they did not define the lang attribute in the root element. More specifically, four pages were from the EN subsample and five were from the ES one.
Figure 7.1 shows the percentage of pages per language that met the three compliance criteria defined in Section 7.3.3.1. In the case of the EN subsample, the results of the human and the automatic evaluation are identical: seventy out of the seventy-four pages (95 percent) featured the lang attribute with a valid value.
By contrast, the results from the manual inspection on the ES subsample do differ from the automated ones. Only forty-four of the seventy-four pages (59 percent) met this success criterion in the ES subsample (see Figure 7.1). The errors discovered during the human evaluation were related to the specific value used in the lang attribute, which did not match the language of the page (es). As previously explained, in the manual inspection, we not only checked the presence and validity of the lang attribute (criteria 1 and 2), but also examined whether the value of that attribute matched the language of the pages from that subset (criterion 3). Table 7.2 displays the breakdown of the values found in the lang attributes. In the case of the Spanish subsample, seventeen pages defined the language of the page as English (en) and eight as English from the US (en-US). These results allow us to answer research question 1.1, since we could argue that the localized subsample of web pages was after all less accessible than the original one, as we initially hypothesized.
Table 7.2 Language values (total number) used in our web sample
| en | en-US | en-GB | es | es-ES | es-US | es-MX | |
|---|---|---|---|---|---|---|---|
| EN pages | 51 | 18 | 1 | - | - | - | - |
| ES pages | 17 | 8 | - | 34 | 5 | 4 | 1 |
7.4.2 Title of the Page
The automatic tool identified no errors related to SC 2.4.2 because all the pages in our web sample had a <title> element. Nevertheless, as explained in Section 7.3.3.2, our manual inspection involved a more in-depth examination of the titles. We analyzed all the EN and ES titles independently against the criteria defined, annotating all the errors found in each title. We considered that the title was not valid if at least one of the criteria was not met. Similarly, note that there is only one title per page. Therefore, the data presented in this section refer not only to the percentage of titles in our sample that comply with SC 2.4.2., but also to the percentage of pages that passed this criterion.
The overall results of our manual inspection showed that less than half of the pages (62 of the 148, 42 percent) were compliant with all the criteria defined. Nonetheless, results differ across the two language subsamples, as can be observed in Figure 7.2. While less than half of the EN titles (34 out of 74, 46 percent) contained at least one error, more than half of the ES titles (52 out of 74, 70 percent) contained one or more errors. This data seems to indicate that the titles of the localized versions (ES) were less accessible than the EN titles, thus allowing us to affirmatively answer research question 1.2 and support our initial hypothesis.
Tables 7.3 and 7.4 present an overview of the types of errors found in the titles, as well as examples taken from our web sample. The former shows the results from criteria 1–6, and the latter presents the results concerning criteria 7–9, which we checked only against the titles in the localized Spanish pages.
Table 7.3 Overview of the errors (total number and %) found in the title of the pages, criteria 1–6
| Type of error | EN (74 pages/titles) | ES (63 pages/titles) | Example |
|---|---|---|---|
| 1.Too long | 20 (27%) | 18 (29%) | The American Kidney Fund (AKF) fights kidney disease on all fronts as the nation’s leading kidney nonprofit. – American Kidney Fund (AKF) |
| 2. It does not identify the subject of the web page | 3 (4%) | 5 (8%) | home |
| 3. It does not make sense when read out of context | 3 (4%) | 7 (11%) | Organ Donor | Organ Donor |
| 4. Repetition | 3 (4%) | 5 (8%) | Medicare.gov: the official US government site for Medicare | Medicare |
| 5. Abbreviation without expanded form | 11 (15%) | 12 (19%) | Home | NIDCD |
| 6. URL | 6 (8%) | 8 (13%) | | FEMA.gov |
Table 7.4 Overview of the errors (total number and %) found in the title of the pages in the ES subsample, criteria 7–9
| Type of error | ES (74 pages/titles) | Example |
|---|---|---|
| 7. Same as in EN | 8 (11%) | Home – National Osteoporosis Foundation |
| 8. Text in EN | 11 (15%) | HOME – Spanish Office of Minority health |
| 9. Combination of EN and ES | 7 (9%) | En Español| Genetic and Rare Diseases Information Center (GARD) – an NCATS Program |
Importantly, in the ES subsample, 15 percent (11 out of 74) of the titles were left in English: they were either the same as their equivalent titles in the EN subsample, or had been changed but were still written in English. In the context of our study, those titles were considered erroneous since they were not written in Spanish, so they were not further analyzed against criteria 1 to 6. Hence, the data we present below account for that dissimilarity. In other words, they present the total number of occurrences as well as the percentage according to the total number of titles analyzed in each subsample, that is, 137 pages: seventy-four in the EN subsample and sixty-three in the ES subsample. Let us now summarize the results per criterion verified:
1. The title is short. We found that the titles of 38 of the 137 analyzed pages (28 percent) had more than 64 characters: 20 of the 74 titles in EN (27 percent) and 18 of the 63 analyzed titles (29 percent) in ES. As can be observed in the example provided in Table 7.3, that was the case of the home page of the American Kidney Fund in English, whose title was comprised of 137 characters.
2. The title identifies the subject of the web page. We found eight titles (6 percent) that did not meet this criterion: three in EN (4 percent) and five in ES (8 percent). For example, the title of the home page of the Life Options Rehabilitation Program in English was just “home.”
3. The title makes sense when read out of context. The titles of ten pages (7 percent) violated this criterion. This error was more recurrent in the titles of the ES pages (N = 7, 11 percent) than in the EN pages (N = 3, 4 percent). When analyzing this criterion, we realized it was strongly related to the previous one (criterion 2). For instance, we interpreted that the previous example “home” did not identify the subject of the page, nor did it make sense when read out of context. In Table 7.3, we provide another example of a title that was not compliant with this criterion: the title of the National Organ and Tissue Donation Initiative in English was “Organ Donor | Organ Donor”. Although a semantic relationship can be established between the title and the subject of the page, we considered that a user would not be able to identify that the title refers to that organization when read out of context.
4. The title does not include repetitions. Eight titles (6 percent) did not meet this criterion: three in EN (4 percent) and five in ES (8 percent). Table 7.3 includes three examples of titles with these unnecessary repetitions. The first one is from the home page in EN of the American Kidney Fund; the second is the from the home page in EN of the National Organ and Tissue Donation; and the third is in the title of the home page of Medicare in EN.
5. The title does not include abbreviations without the expanded form. This was one of the most common criterion violations found in our web sample, as 23 pages (17 percent) had an abbreviation without the expanded form in the title: 11 in EN (15 percent) and 12 in ES (19 percent). For example, the title of the home page in EN of the National Institute on Deafness and Other Communication Disorders was “Home | NIDCD.” People unfamiliar with this organization might not be able to decode the acronym by just reading the title of the home page, especially when read out of context or through assistive technology.
6. The title does not include URL addresses. We found fourteen titles (10 percent) that did not meet this criterion: six in EN (8 percent) and eight in ES (13 percent). For example, the Federal Emergency Management Agency in both the EN and ES home pages used the title “| FEMA.gov.” Again, people who know neither those organizations nor the URLs of the official websites might be unable to identify the subject of the page with such a title.
All in all, the data suggests that there were more SC 2.4.1 violations in the pages of the ES subsample than in the EN ones. The two most common errors were: a) the excessive length of the titles, and b) the presence of abbreviations without an expanded form. We now go on to describe in detail the three criteria that were analyzed only in the ES subsample (see Table 7.4):
7. The localized title differs from the original title. Ten pages in our ES subsample had the same title as in the EN page, but we considered that only eight (11 percent) violated this principle. This dissimilarity was due to the exception we made concerning the titles that included only the name of the organization. This was the reasoning in these particular cases:
a) If the name was in English and no official Spanish translation was proposed and used by the organization in the home page, the title was considered correct. We found two pages whose titles met this condition. We contend that not translating the name of the organization is completely valid, especially if it is treated as a proper noun.
b) If the name was in English within the title but it appeared in Spanish in the body of the localized web page, we marked it as an error. That was the case, for example, of the National Osteoporosis Foundation, that used in the ES home page the same title as in the EN page: “Home National Osteoporosis Foundation.” However, in the main content of the ES version, users could find the corresponding name of the organization in ES: “Fundación Nacional de la Osteoporosis.”
8. The text is in the target language. The titles of eleven pages (15 percent) from our ES subsample did not meet this criterion. We included in this category titles that were left in English and considered incorrect (i.e., the eight titles that did not meet criterion 7) and three additional titles that were different from the EN original title but were left in English. That was the case of the home page in ES of the Office of Minority Health: in the title “HOME – Spanish Office of Minority health,” the text was different from that of the original EN page “Home Page – Office of Minority Health (OMH),” but still in the source language. In addition, the name of the organization in Spanish was included in the footer: “Oficina de Salud de Minorías.”
9. The title is not composed of text in both the original and the target languages. Seven pages (9 percent) from our ES subsample violated this criterion because they included a combination of English and Spanish in their titles. This occurred, for example, on the website of the Genetic and Rare Diseases Information Center (GARD): “En Español | Genetic and Rare Diseases Information Center (GARD) – an NCATS Program.”Footnote 20 Screen readers, when reading aloud, normally pronounce in only one language by default. A title like this one could be difficult to understand, since part of the sentence would be mispronounced. Further research with screen reader users would be needed to fully understand whether such bilingual strings actually represent a real accessibility obstacle.
7.5 Discussion and Conclusions
Access to health information is crucial in today’s society. This study set out to investigate the accessibility of health websites from a localization perspective. We gathered a web sample of seventy-four multilingual websites of health organizations mainly based in the US, assuming that EN was the source language and ES the target. Then we used automatic and human evaluation methods to check the home page of both language versions against two localization-related WCAG success criteria: 2.4.2, Page titled and 3.1.1, Language of the page.
The results provided by the automatic tool showed that the overall accessibility of the assessed web pages was generally high, and no noteworthy differences were observed between the two language versions. At first, this seemed to refute our initial hypothesis, as we expected the localized subsample to be less accessible than the original one. However, our manual inspection evaluation analysis revealed several underlying accessibility barriers with regard to the two success criteria analyzed in the target version. Our results are consistent with those of Reference Jiménez-CrespoJiménez-Crespo (2008), Reference MinacapilliMinacapilli (2018), and Reference PontusPontus (2019) with regard to the lower accessibility level observed in localized websites when compared to the original ones – first, with respect to general accessibility scores and secondly, in relation to the appropriateness of the page language definition and title.
The fact that some titles and language values were not modified in the localized pages suggests that these aspects might have been overlooked during localization. We hypothesize that this oversight could be due to some combination of the following factors:
a) Lack of localization knowledge. The localization process entails not only the translation of the textual content of the page, but also the modification of other technical and cultural aspects, such as the adaptation of the two accessibility features studied. Localization agents might have simply overlooked the need to modify the target page title or its language definition, whether unintentionally or due to lack of experience and know-how.
b) Lack of accessibility knowledge. We contend that accessibility assurance should be an inherent step in the localization process. Agents involved in the development of the examined web sample might have been unaware of accessibility guidelines and recommendations, and may therefore have overlooked implementation of the best practices we investigated. Thus we stress the importance of including accessibility study in the curricula of translators and localizers (Reference Torres-del-Rey and Rodríguez VázquezTorres-del-Rey and Rodríguez Vázquez, 2016) or in the current localization data exchange standards (Reference Torres-del-Rey and Morado VázquezTorres-del-Rey and Morado Vázquez, 2019).
c) Varied access to web authoring tool settings. Localization workflows vary depending on the number of agents involved in the process, their professional profiles, and the available tools, among other aspects. Sometimes, when working directly on web authoring tools, translators or localizers have access only to the main content of the web page, and not to all the metadata that should be adapted, which might be hidden and/or protected. For example, in modern Content Management Systems, both the language and the title of a page are typically defined in a specific “properties” section, not on the page editor where the main content of the page can be modified. Inability to reach those particular sections due to lack of expertise or limited access rights could have contributed to the failure to make all the necessary changes in the ES versions. In other workflows, localizable content is extracted from the web authoring tool and brought into a localization data exchange file, such as XLIFF (Reference Torres-del-Rey and Morado VázquezTorres-del-Rey and Morado Vázquez, 2015). Again, depending on the tool settings, all localizable information might or might not be contained therein. In any case, we believe that well-trained localization specialists with accessibility knowledge would have been able to identify the issues related to SC 2.4.2 and SC 3.1.1 and to propose appropriate solutions.
7.5.1 Challenges in Localization-Oriented Accessibility Evaluation
Our study helped us identify a number of challenges related to the application of existing accessibility evaluation methods for the assessment of localized websites. With no claim to be exhaustive, we list some of them in this section, particularly in relation to the success criteria which were our focus.
7.5.1.1 Automated Audits
As mentioned in Section 7.3, and demonstrated through the results presented in Section 7.4, the automated tool used in this study was not able to check all the multilingual accessibility compliance criteria defined. Firstly, with respect to the language of the page, the tool was unable to identify the mismatch between the value of the lang attribute and the actual language of the web pages. While this error was absent in the EN subsample, manual inspection revealed that certain pages in the ES subsample did violate this criterion. Secondly, with regard to the title of the page, only its presence could be verified; the other nine defined criteria had to be assessed through manual inspection. These divergences between automated and human evaluation methods were also observed in prior work. For instance, Hanson and Richards (2013: 19), in a WA study combining automated and human evaluation, did indeed find that manual inspection revealed more errors than were detected by automated evaluation.
Our results reinforce the well-stablished idea that, when evaluating WA, it is paramount to complement the use of automated tools with manual inspection of the pages being studied (Reference Abou-Zahra, Harper and YesiladaAbou-Zahra, 2008; Reference Brajnik, Hartmann, Zhou and KirchbergBrajnik, 2008). However, this point becomes even more crucial for multilingual WA assessment. As we have already noted in previous studies (Reference Rodríguez VázquezRodríguez Vázquez, 2016b), there is still room for improvement in automated accessibility evaluation tools with regard to the verification of language-related aspects. Some of the additional audits that could be implemented to facilitate localization-oriented accessibility evaluation are:
1. Concerning the language of the page, the tool could feature an audit through which the human language used in the content of the page is recognizedFootnote 21 and later compared with the value used in the lang attribute.
2. Regarding the title of the page, the tool could also integrate some of the compliance criteria we defined, so that it could: a) automatically measure the title’s length (criterion 1); or b) by using regular expressions, detect repeated content (criterion 4), abbreviations without expanded form (criterion 5), or URL addresses (criterion 6). In addition, the technology used in existing CAT tools (such as quality assessment modules including spell checkers) could be integrated to check criteria 7, 8, and 9.
7.5.1.2 Definition of Compliance Criteria
For the purposes of our study, we defined and applied a specific list of language-oriented accessibility compliance criteria. Our ultimate goal was to complement other lists of criteria already defined in existing resources – for instance, in automated tools or in the test rule set of the W3C (W3C Web Accessibility Initiative, 2022) – in order to add localization-oriented value. In the process, however, we have also identified several challenges that deserve further consideration in future investigations.
The criteria selected to assess page titles were based mainly on official recommendations, as explained in Section 7.3.2.2. However, to the best of our knowledge, those recommendations do not stem from empirical research; hence the need to conduct more studies on their actual accessibility impact. Further, most of the existing recommendations are open for interpretation. For example, the recommendation for the length of titles is described by the WAI only as “short.” To establish a length threshold in our study that would enable us to objectively measure this criterion’s compliance, we used the 64-character limit recommended by Tim Reference Berners-LeeBerners-Lee (1992) in his Style Guide for online hypertext. However, it would be worthwhile to explore the possibility of defining a more flexible rule that would acknowledge that different languages might tolerate different limits.Footnote 22
As a related matter, the geographical and cultural context of the sites’ target audience should also be considered when verifying web page titles. For example, acronyms could be considered in titles if they are well known by the target audience and therefore do not pose accessibility barriers. With due recognition for the universal nature of the Web, in these particular cases, it would be ideal during human evaluation to include accessibility experts or end users living in the relevant geographical context to help decide on the best strategy. For health websites, this approach could be followed for local nonprofit or community association portals. For international websites, however, its validity should be scientifically measured.
All in all, we believe that web developers, localizers, and accessibility experts could benefit from a single resource that would assemble these and other recommendations for multilingual WA assessment. Our current list of nine criteria could serve as a starting point for building that cohesive resource, and could pave the way for future empirical studies in which the impact of the final list could be tested with real users.
7.5.1.3 Need for Accessibility Enablers with an Interdisciplinary Background
We firmly believe that localization professionals could play a leading role in making a website accessible, not only by ensuring the accessibility of the final localized web product but also by revealing and resolving compliance issues in the original version. Both of the researchers involved in the study presented here are web localization and accessibility experts. However, in a non-academic context, more human resources would probably have been needed to conduct a multilingual accessibility evaluation – at least a web developer and an accessibility expert per language version. These requirements could imply more complex workflows and increased costs for websites with more than two language versions.
In this sense, advocacy for interdisciplinary training is essential. In the last decade, several members of the Cod.eX research group have put forward various proposals for the inclusion of accessibility in the curricula of translation and localization training programs (Reference Torres-del-Rey and Rodríguez VázquezTorres-del-Rey and Rodríguez Vázquez, 2016; Reference Torres-del-Rey, Sawyer, Austermühl and Enríquez RaídoTorres-del-Rey 2019). Similarly, we have suggested integration of accessibility best practices and evaluation procedures in the quality assurance phase of the localization process (Reference Rodríguez Vázquez, O’Brien, Antona and StephanidisRodríguez Vázquez and O’Brien, 2017; Reference Torres-del-Rey and Morado VázquezTorres-del-Rey and Morado Vázquez, 2019). It is true that, in an ideal scenario, accessibility should not be treated as an afterthought but as in integral step in web development and localization processes. However, accessibility audits performed before the launch of the localized web product would contribute to the detection and avoidance of the compliance violations that we observed in our study, among others. This strategy requires not only continuing technological progress, so that new types of audits concerning language-related accessibility features can be integrated, but also understanding of the need for accessibility enablers with interdisciplinary backgrounds.
7.5.2 Limitations and Future Work
As explained at the beginning of Section 7.5, the present study extends current knowledge of multilingual WA. However, we are aware that our findings resist generalization: our web sample was limited in terms of genre (health websites), language combination (English and Spanish), and geographical context (US and International organizations), among others. Moreover, our study focused upon only two aspects that can affect the accessibility of a web page. In order to obtain a comprehensive picture of the accessibility of multilingual health websites, a broader evaluation should be conducted – for instance, by (a) examining the quality of the multilingual health information included in the site; (b) verifying other success criteria; or (c) involving members of selected target population groups (e.g., migrants or screen reader users) in the human evaluation of the two accessibility features studied. For instance, it would be worth exploring whether the use of different user agents (including browsers and assistive technologies) impacts the acceptability of page titles, particularly concerning limitations in the number of characters (i.e., of title length) or the combined use of languages (i.e., of multilingual titles).
Apart from the ideas for future work already shared in previous sections, another potentially fruitful avenue for new research is the study of similar websites from other geographical contexts and with other language combinations. We have studied websites mainly from US organizations. Further studies could include, for example, multilingual websites from organizations based in Spanish-speaking countries, in order to explore whether similar accessibility violations are observed. Similarly, a follow-up study could examine our web sample in greater depth. For instance, it would be interesting to consider multiple Spanish variants, including “International Spanish” (Reference Jiménez-CrespoJiménez-Crespo, 2010) and to study the various localization strategies adopted, as per Reference YunkerYunker (2003), to deal with varied Spanish-speaking end users. We could also examine the websites of organizations based in the states with the highest concentration of Hispanics, as in prior work with US web corpora (Reference Jiménez-CrespoJiménez-Crespo, 2012), and could investigate whether the accessibility level of the localized Spanish pages is higher when compared with other sites in the sample.
In addition, further studies could examine the influence of web authoring tools in the establishment of certain accessibility best practices, including those we have analyzed: the language and title of web pages. As we have hypothesized, the lack of access to certain advanced features might impact the adaptation of those metadata elements. However, other factors might also condition the final content rendering. For example, to create the title of a given page, certain tools recommend combining the general name of the website with that of the page to be described.Footnote 23 As we have observed, for home pages, this procedure could yield redundant titles or repetitions.
In conclusion, we have attempted to demonstrate that localization specialists can play a central role in identifying and resolving accessibility issues to produce a more accessible multilingual web for all. Our findings and suggestions will, we hope, help others to explore new ways of studying multilingual accessibility and to better understand the causes of imbalances between original and localized website versions.
7.6 Acknowledgments
The preliminary results of our study were presented at the 7th International IATIS Conference. Barcelona, Spain, September 14–17, 2021. We would like to thank the organizers of our panel for the valuable feedback provided.
Open access