Hostname: page-component-78c5997874-dh8gc Total loading time: 0 Render date: 2024-11-17T14:08:18.294Z Has data issue: false hasContentIssue false

Does neurocognition contribute to age-related deficits in the online navigation of electronic patient health portals?

Published online by Cambridge University Press:  09 February 2023

Anastasia Matchanova
Affiliation:
Department of Psychology, University of Houston, Houston, USA
Michelle A. Babicz
Affiliation:
Department of Psychology, University of Houston, Houston, USA Mental Health and Behavioral Services, James A. Haley Veterans’ Hospital, Tampa, USA
Victoria M. Kordovski
Affiliation:
Department of Psychology, University of Houston, Houston, USA Department of Psychiatry & Behavioral Sciences, Johns Hopkins University, School of Medicine, Baltimore, USA
Savanna M. Tierney
Affiliation:
Department of Psychology, University of Houston, Houston, USA Mental Health Care Line, Michael E. DeBakey Veterans Affairs Medical Center, Houston, USA
Samina Rahman
Affiliation:
Department of Psychology, University of Houston, Houston, USA Department of Psychology, Washington State University, Washington, USA
Luis D. Medina
Affiliation:
Department of Psychology, University of Houston, Houston, USA
Clint Cushman
Affiliation:
Department of Psychiatry, University of California, San Diego, USA
Steven Paul Woods*
Affiliation:
Department of Psychology, University of Houston, Houston, USA
*
Corresponding author: Steven Paul Woods, email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Objective:

The internet serves an increasingly critical role in how older adults manage their personal health. Electronic patient portals, for example, provide a centralized platform for older adults to access lab results, manage prescriptions and appointments, and communicate with providers. This study examined whether neurocognition mediates the effect of older age on electronic patient portal navigation.

Method:

Forty-nine younger (18–35 years) and 35 older adults (50–75 years) completed the Test of Online Health Records Navigation (TOHRN), which is an experimenter-controlled website on which participants were asked to log-in, review laboratory results, read provider messages, and schedule an appointment. Participants also completed a neuropsychological battery, self-report questionnaires, and measures of health literacy and functional capacity.

Results:

Mediation analyses revealed a significant indirect effect of older age on lower TOHRN accuracy, which was fully mediated by the total cognitive composite.

Conclusions:

Findings indicate that neurocognition may help explain some of the variance in age-related difficulties navigating electronic patient health portals. Future studies might examine the possible benefits of both structural (e.g., human factors web design enhancement) and individual (e.g., training and compensation) cognitive supports to improve the navigability of electronic patient health portals for older adults.

Type
Research Article
Copyright
Copyright © INS. Published by Cambridge University Press, 2023

The vast majority of US healthcare systems offer patients online access to their electronic health records (Henry et al., Reference Henry, Barker and Kachay2019). These online portals allow patients to view visit summaries, message providers, schedule appointments, and refill medications (Department of Health and Human Services, 2017). Use of online patient portals is associated with greater health knowledge, self-efficacy and decision-making, and positive health behaviors (Han et al., Reference Han, Gleason, Sun, Miller, Kang, Chow, Anderson, Nagy and Bauer2019). Yet only 15–30% of patients report using even a single feature of these online tools (Hong et al., Reference Hong, Jiang and Liu2020). Patients that use online portals are more likely to be White women with higher levels of education and income (Hong et al., Reference Hong, Jiang and Liu2020). Barriers to online patient portal use include limited access to high-speed internet, less experience with online communication, and lower health literacy (Coughlin et al., Reference Coughlin, Stewart, Young, Heboyan and De Leo2018).

Although older adults were historically unlikely to use online patient portals (Hong et al., Reference Hong, Jiang and Liu2020), they now comprise nearly half of all users (Wildenbos et al., Reference Wildenbos, Maasri, Jaspers and Peute2018). While older adults are well represented among online healthcare portal users, they may nevertheless have difficulties navigating such tools independently. Qualitative studies point to two key barriers: (1) privacy and security concerns; and (2) access and ability to use technology (Sakaguchi-Tang et al., Reference Sakaguchi-Tang, Bosold, Choi and Turner2017). Indeed, older age is reliably associated with poorer internet search and navigation skills at small-to-medium effect sizes (Agree et al., Reference Agree, King, Castro, Wiley and Borzekowski2015). Age-related neurocognitive decline may partly explain older adults’ difficulties navigating online patient health portals. Declines in cognition (particularly in the domains of working memory, executive functions, memory, and visuomotor processing speed; Wecker et al., Reference Wecker, Kramer, Wisniewski, Delis and Kaplan2000), can interfere with many different activities of daily living (e.g., Tucker-Drob, Reference Tucker-Drob2011), including health behaviors (e.g., Park, Reference Park, Craik and Salthouse1992).

Internet navigation can place demands on several brain networks and neurocognitive abilities that commonly decline with age (e.g., Small et al., Reference Small, Moody, Siddarth and Bookheimer2009). At face value, simply operating a computer involves visuomotor processing speed to efficiently navigate the contents on a page using a keyboard and mouse, memory for recall of passwords and search terms, and executive functions for planning and carrying out specific goals or tasks online. Research shows medium-to-large relationships between the accuracy of health-related internet searches and several aspects of neurocognition, including executive functions, memory, and visuomotor processing speed (Woods et al., Reference Woods, Kordovski, Tierney and Babicz2019). Since most of these domains are sensitive to age-related cognitive decline, it is possible that neurocognitive abilities might partially explain the relationship between older age and poorer internet navigation skills (Czaja et al., Reference Czaja, Sharit, Ownby, Roth and Nair2001). Consistent with this hypothesis, previous studies have demonstrated that the effects of older age on internet search task performance were dampened when controlling for neurocognition (Chevalier et al., Reference Chevalier, Dommes and Marquié2015). Therefore, older age may lead to declines in neurocognition that can also impact effective navigation of online patient portals.

We are aware of only one prior study examining the role of neurocognition in navigating online patient health portals. In 2016, Woods et al. reported on 46 middle-aged adults with HIV and 21 seronegative participants who completed the Test of Online Health Records Navigation (TOHRN), which is an experimenter-controlled task designed to simulate an online patient health portal. Participants logged into TOHRN, read a message from a healthcare provider, checked their lab results, and scheduled a follow-up appointment. Participants with HIV-associated neurocognitive disorders (HAND) had the lowest TOHRN accuracy scores at large effect sizes. In the HIV sample, lower TOHRN accuracy was associated with poorer performance on measures of memory and executive functions, as well as with higher levels of viremia and lower health literacy. Thus, it is reasonable to hypothesize that age-related cognitive declines may interfere with online patient portal use among older adults. The current study therefore extends the literature in two important ways. First, we evaluate the hypothesis that older age is associated with increased difficulty in navigating an online, performance-based patient health portal. Second, we examine the mediating role of neurocognitive ability in the association between older age and poorer online patient portal performance.

Methods

Participants

Study participants were drawn from 97 adults recruited from the Houston area between March 2019 and March 2020. The parent study examined the neurocognitive aspects of age-related difficulties in different dimensions of internet navigation and was approved by the institutional review board. Study participants were recruited into two discrepant age groups. All procedures performed involving human participants were in accordance with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Fifty-two younger adults (age 18–35 years) were recruited from the student body of the University of Houston via the student research management system or from the community by word-of-mouth. Forty-five older adults (age 50+) were recruited through online postings (e.g., Craigslist), community sites (e.g., libraries), and word-of-mouth. Participants who were recruited from the university system received research credit, while community participants received a gift card and an optional written summary of their normative cognitive performance. Data from the parent study are published elsewhere (e.g., Kordovski et al., Reference Kordovski, Tierney, Rahman, Medina, Babicz, Yoshida, Holcomb, Cushman and Woods2021; Rahman et al., Reference Rahman, Kordovski, Tierney and Woods2021; Tierney et al., Reference Tierney, Kordovski, Rahman, Medina, Damian, Collins and Woods2022), but the electronic patient portal data have not previously been reported.

Inclusion criteria were adequate English language proficiency to participate in neurocognitive testing, at least 1.5 hr of internet use per weekFootnote 1 (Choi & DiNitto, Reference Choi and DiNitto2013), and capacity to provide informed consent. All participants obtained passing scores on an embedded test of cognitive performance validity (Schroeder et al., Reference Schroeder, Twumasi-Ankrah, Baade and Marshall2012) and had adequate vision, hearing, and basic motor function. We excluded participants with histories of major neurological disorders, severe psychiatric conditions, and active substance use disorders. We also excluded anyone with impaired age- and education-adjusted normative scores (i.e., -1.5 Z-score cutoff; Rossetti et al., Reference Rossetti, Lacritz, Cullum and Weiner2011) on the Montreal Cognitive Assessment (MoCA; Nasreddine et al., Reference Nasreddine, Phillips, Bédirian, Charbonneau, Whitehead, Collin, Cummings and Chertkow2005). Two younger adults and seven older adults did not complete the electronic health portal navigation tasks due to self-discontinuation or technical administration issues. The final analyzable sample included 49 younger (range 18–32 years) and 35 older (range 51–75 years) participants whose characteristics are shown in Table 1.

Table 1. Descriptive characteristics of the older and younger study groups

Note. Data represent M (SD) [Sample range] or %; eHEALS = Electronic Health Literacy Scale; MoCA = Montreal Cognitive Assessment; SCQ = Self-Administered Comorbidity Questionnaire; WRAT-4 = Wide Range Achievement Test, Fourth Edition.

a As measured by DSM-5 cross-cutting symptoms questionnaire in the DSM-5 (Diagnostic and Statistical Manual).

b Per speedtest.net.

c A composite TOHRN score was created by summing the total accuracy scores from standard and alternate versions (alpha = .869, range = 6–22).

Materials and methods

The assessment consisted of a standardized neuropsychological battery, self-report questionnaires, measures of health literacy and functional capacity, and several internet navigation tasks. Research assistants assessed each participant in-person during a single session.

Electronic patient portal assessments

Test of online health records navigation (TOHRN)

The TOHRN is an experimenter-controlled website designed to simulate a typical electronic patient portal interface (Woods et al., Reference Woods, Iudicello, Morgan, Cameron, Doyle, Smith and Cushman2016). Participants were tested using one of several Dell PCs running Windows 10 at the recommended display settings, using a wired internet connection and wired laser mouse. The TOHRN website contains information about a network of providers, as well as different sections to which participants could find information on lab results, current prescriptions and diagnoses, and messages from various healthcare entities. The website gave participants access to six tabs in the navigation bar at the top of the home page: Home, My Account, Appointment, My Medical Records, Message Center, and Logout. Participants began by logging in to their electronic health record using a mock username and password. Participants were provided with the following instructions: “(1) There are several unread messages in your message center. Log onto the website and read your messages; (2) Be sure to look for any messages regarding test results; and (3) Check your lab results. If you receive abnormal test results, follow the instructions your doctor left in their note to you.” Participants were read the task instructions aloud by the examiner and provided with a hardcopy list of specific tasks they were charged with completing, which remained visible to participants for the duration of the examination. No examiner assistance or prompts were provided at any time.

To complete the task accurately, the participant must log in successfully, locate the specific message in their message center and open it, observe that the test results are abnormal, and note that the doctor provides instructions in a message to schedule a follow-up appointment in 30 days. To follow the provider’s instructions, the participant must select the appropriate appointment from a set of available slots displayed on a calendar in the Appointments section. Participants received one point for each of these step components that they completed (Cronbach’s alpha = 0.791), which were summed to create a total score for which higher values reflect better performance (sample range = 3–11; maximum possible = 11). We also calculated total completion time, which was manually recorded immediately after the instructions were read and stopped after the participant stated they were finished.

An alternate version of the TOHRN task was also administered in a counterbalanced manner with approximately 2.5 hr in between administrations. The alternate version was similar in structure to the standard version but differed in the specific message content, provider information, and scheduling details. Participants received one point for each accurately completed step (alpha = 0.808), which were summed to create a total score for which higher values reflect better performance (sample range = 2–11; maximum possible = 11). The accuracy scores from standard and alternate versions showed a large correlation (r s = .52) and did not differ significantly in the full sample (p = .461). None of these cross-form associations varied meaningfully by age group (ps < .05). In order to minimize Type 1 error, a composite TOHRN score was created by summing the total accuracy scores from standard and alternate versions (alpha = .869, range = 6–22). Note that findings reported below did not differ if the individual TOHRN accuracy forms were used instead of the composite score.

With regard to completion time scores for the TOHRN, we did not see any significant overall differences in the full sample between performance on the standard (237 ± 171 s) and alternate (201 ± 133) versions (p = .096). Yet, these two TOHRN completion time scores were only weakly correlated (r s = .17, p = .114). As such, we do not report any further analyses related to TOHRN completion time.

Self-report questionnaires

Participants completed a 6-item scale measuring their perceived self-efficacy using electronic patient portals (Davis, Reference Davis1989; e.g., “I find my online healthcare management website easy to use”). Each item was rated on a scale from 0 (strongly disagree) to 4 (strongly agree). A total score (alpha = .950) was derived by summing the 6 items (sample range = 0–24), with higher scores reflecting greater levels of self-efficacy. We also adapted a single item from Davis (Reference Davis1989) to measure the frequency with which participants used electronic patient portals in their daily lives, which ranged from 1 (not at all) to 6 (several times per day). The sample range was 1–4 (several times per week).

Neuropsychological assessment

The neuropsychological battery tests executive functions, attention, verbal memory, and visuomotor processing speed. Executive functions were measured using the 20-Questions subtest (abstraction score) of the Delis-Kaplan Executive Function System (D-KEFS; Delis et al., Reference Delis, Kaplan and Kramer2001), Condition 4 (Letter-Number Sequencing) of the D-KEFS Trail Making Test (TMT) completion time, and the Action (Verb) Fluency test (Piatt et al., Reference Piatt, Fields, Paolo, Koller and Tröster1999; Woods et al., Reference Woods, Scott, Sires, Grant, Heaton and Tröster2005). Attention was measured using the Digit Span Total score of the Weschler Adult Intelligence Scale – IV (WAIS-IV; Wechsler, Reference Wechsler2008) and a timed 30-s version of the Serial Sevens Test (Manning, Reference Manning1982). Trials 1–4, Short Delay Free Recall, and Long Delay Free Recall of the California Verbal Learning Test-Second Edition Short Form (CVLT-II SF; Delis et al., Reference Delis, Kramer, Kaplan and Ober2000) were used to measure verbal memory. Visuomotor processing speed was measured using Conditions 1–3 and Condition 5 of the D-KEFS TMT. Raw total scores of these neuropsychological tests were converted to sample-based Z scores.

Sample characterization

Participants completed a demographics form, the Self-Administered Comorbidity Questionnaire (SCQ; Sangha et al., Reference Sangha, Stucki, Liang, Fossel and Katz2003), the Level-1 Cross-Cutting Symptom measure from the Diagnostic and Statistical Manual of Mental Disorders (5th Ed.; American Psychiatric Association, 2013), and the reading subtest of the Wide Range Achievement Test-Version 4 (WRAT-4; Wilkinson & Robertson, Reference Wilkinson and Robertson2006). General internet use quantity and frequency over the past 30 days was measured with three items (Baggio et al., Reference Baggio, Iglesias, Berchtold and Suris2017) that were collapsed into a single score (range = 0–63), with higher scores indicating more use. Internet anxiety was measured with 6 questions drawn from Joiner et al. (Reference Joiner, Brosnan, Duffield, Gavin and Maras2007; e.g., “I always feel anxious when using the internet”). These items were rated on a five-point scale from 0 (strongly disagree) to 4 (strongly agree) and were summed (range = 0–24), with higher scores reflecting greater anxiety. General health literacy was measured with the Expanded Numeracy Scale (Lipkus et al., Reference Lipkus, Samsa and Rimer2001), the Newest Vital Sign (Weiss et al., Reference Weiss, Mays, Martz, Castro, DeWalt, Pignone, Mockbee and Hale2005), and the eHealth Literacy Scale (eHEALS; Norman & Skinner, Reference Norman and Skinner2006).

Data analysis

All analyses were conducted using SPSS (version 25.0). Visual inspection and screening of the data were conducted to ensure accuracy, identify outliers, and detect missing data (Van den Broeck et al., Reference Van den Broeck, Argeseanu Cunningham, Eeckels and Herbst2005). The few outliers that existed were Winsorized (Tabachnick & Fidell, Reference Tabachnick and Fidell2013), which reduced skew to within acceptable levels. TOHRN variables were non-normally distributed (Shapiro–Wilk W ps < .0001), so nonparametric tests (e.g., Spearman’s rho) were used for univariable analyses wherever possible. Principal component analysis (PCA; SPSS version 25) was first used to develop a total cognitive composite score from the sample-based neurocognitive Z scores. Factor loadings ≥ 0.40 were considered significant for individual items (Floyd & Widaman, Reference Floyd and Widaman1995) and eigenvalues ≥ 1.0 were considered significant for a factor (Kaiser, Reference Kaiser1960). The optimal number of components was determined by scree plots and parallel analysis to compare the components to simulated chance values (Glorfeld, Reference Glorfeld1995; O’Connor, Reference O’Connor2000). Data were inspected prior to the analysis to ensure that the following assumptions are met: (1) univariate normality; (2) each factor comprised of >3 variables; (3) the ratio of respondents to variables is a minimum 5:1; (4) the correlation between the variables is >0.30; (5) any missing data are at random; and (6) there is no multicollinearity or singularity (Field et al., Reference Field, Miles and Field2012; Yong & Pearce, Reference Yong and Pearce2013).

The main study hypotheses were investigated with a mediation model (MacKinnon-et al., Reference MacKinnon, Fairchild and Fritz2007) using the PROCESS module for SPSS (Hayes, Reference Hayes2013). The model included age as the independent variable, the total cognitive PCA composite score as the mediator, and TOHRN accuracy as the dependent variable. PROCESS uses an observed variable ordinary least squares regression-based path analysis. All models were interpreted using maximum likelihood estimation and 95th percentile bootstrapped confidence intervals, which allows for non-normal data. Effect sizes were interpreted as the percent of the total effect (c-path) that is accounted for by the indirect effect (a x b). If the direct effect (c’-path) is larger than the total effect, the absolute value of the total value of the direct path will be used to calculate effect size. Effects were not considered significant if the 95% CI contained zero. Of note, for dichotomous independent variables, the PROCESS module generates standardized regression coefficients in partially standardized form. For the proposed model, given large effect size of α, a medium effect size of ß, a c’ value of .14 (which indicates a partially mediated model), the sample size required to conduct a mediational analysis with .8 statistical power would be 59 participants (Fritz & MacKinnon, Reference Fritz and MacKinnon2008).

Results

Preliminary analyses

Factor structure of the neurocognitive measures

The PCA revealed that all the cognitive measures except D-KEFS 20-Questions, WAIS-IV Digit Span Total, and the Serial Sevens Test significantly loaded onto a single factor (See Table 2). As such, these three measures were excluded from the total cognitive composite factor scores and all primary and exploratory analyses. After removing the three measures mentioned above, the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy was 0.771, suggesting acceptable sampling and utility of the PCA for the purposes of the analysis (Kaiser, Reference Kaiser1960). Bartlett’s test of sphericity was significant, approximate X 2(36) = 372.3, p < .001, suggesting sufficient relation between variables to detect an underlying component structure. The remaining nine Z scores loaded on to a single component in the PCA analysis, which accounted for 32.2% of the total variance (alpha = .756). Thus, the factor scores from this single component of neurocognitive functioning were used for the primary analyses. See Supplementary Table 1 for primary cognitive variables in the older and younger study groups.

Table 2. Total cognitive composite principal component analysis (PCA) loadings

Note. Data represent component loading scores; D-KEFS = Delis-Kaplan Executive Function System; TMT = Trail Making Test; WAIS-IV = Weschler Adult Intelligence Scale – IV; CVLT-II SF = California Verbal Learning Test-Second Edition Short Form; SDFR = Short Delay Free Recall; LDFR = Long Delay Free Recall.

Kaiser–Meyer–Olkin (KMO) = .752; Bartlett’s test of sphericity: X 2(66) = 424.6; p < .001.

Determining covariates for the primary mediation model

We adopted a confound approach (Field-Fote, Reference Field-Fote2019) to select covariates for the mediation model. Specifically, any descriptive variable in Table 1 that differed significantly by age group and was significantly associated with TOHRN was included as a covariate. This approach was taken to avoid over-saturating the mediation model, which was important given the relatively small sample sizes. The age groups differed on race/ethnicity, WRAT-4 word reading, internet use frequency, internet connection speed, and current psychiatric symptoms (ps < .05), but none of these factors were associated with TOHRN accuracy in the full group or in the separate age groups (all ps > .05). As such, no covariates were included in the models.Footnote 2 See Supplementary Table 2 for a simple correlation matrix of relevant demographic variables.

Primary analyses

Univariable correlations

Table 3 shows the univariable correlation coefficients between age, TOHRN accuracy, and total cognitive PCA factor scores. Older age was significantly associated with lower TOHRN accuracy at a small effect size (p = .013). The total cognitive composite was negatively associated with age (p < .001) and positively associated with TOHRN accuracy (p < .001) at broadly medium effect sizes.

Table 3. Zero-order correlation matrix of all primary variables in the mediation models

Note. The values in this table reflect the Spearman’s correlation between each variable. Test of Online Health Records Navigation = TOHRN.

* Indicates correlation is significant at p < .05 level.

** Indicates correlation is significant at p < .01 level.

a For univariable correlations with a continuous and a dichotomous variable, values in this table reflect point-biserial correlations.

Mediation analyses of age, cognition, and TOHRN accuracy

As shown in Figure 1, the total cognitive Z-score PCA composite was a significant full mediator of the relationship between age and TOHRN accuracy (b = -0.47, 95% CI [-0.73, -0.24]). Findings indicated that older adult participants were expected to obtain 1.77 points lower on TOHRN accuracy than younger adult participants, and this score discrepancy was mediated by the total cognitive composite. After accounting for the effect of the cognitive composite, age did not have a significant direct effect on TOHRN accuracy (b = -0.40, 95% CI [-1.99, 1.18]). Of note, changing the age variable from categorical to continuous did not significantly impact the results of this model.Footnote 3

Figure 1. Mediation model with partially standardized bootstrap estimates and 95% confidence interval values for age group (n = 35 older adults; n = 49 younger adults) and TOHRN accuracy mediated by the total cognitive composite.

Post hoc analyses

Domain-level mediators of the association between age and TOHRN accuracy

Given that the total cognitive composite included tests that assess different domains, we conducted a series of post hoc analyses to examine possible domain-level associations. First, PCAs were conducted to create domain-level composite scores based on a priori groupings of tests of attention, speeded executive functions, visuomotor processing speed, and verbal memory (see Supplementary Table 3). The psychometrics of the attention and speeded executive functions domains were deemed unacceptable for further analysis. Both verbal memory and visuomotor processing speed had acceptable sampling and utility and sufficient relation between variables (KMO range: 0.709–0.760; Bartlett’s test of sphericity: all ps < 0.001). Results showed that both verbal memory (b = -0.14, 95% CI [-0.28,-0.02]) and visuomotor processing speed (b = -0.34, 95% CI [-0.56, 0.09]) were significant full mediators of the relationship between age group and TOHRN accuracy.

Discussion

While older adults have become more likely than younger adults to utilize online healthcare portals (Hong et al., Reference Hong, Jiang and Liu2020), they nevertheless have lower self-efficacy in navigating such systems (e.g., Nahm et al., Reference Nahm, Zhu, Bellantoni, Keldsen, Charters, Russomanno, Rietschel, Son and Smith2020). The present study demonstrated that older adults are less accurate in their performance on an online patient portal navigation task (i.e., TOHRN) as compared to younger adults. That is, older adults had more difficulty navigating a health website to read their lab results, communicate with providers, and schedule an appointment as compared to younger adults. Importantly, these age-related differences were associated with medium effect sizes and were not confounded by relevant sociodemographic (e.g., gender, education) and online factors (e.g., portal use frequency, self-efficacy).

One potential reason for this age-related difference in online patient portal performance is age-related neurocognitive decline. Online health navigation can be complicated and may place high demands on different aspects of cognition that are sensitive to age-related decline, including attention, episodic memory, visuomotor processing speed, and executive functions (Kordovski et al., Reference Kordovski, Tierney, Rahman, Medina, Babicz, Yoshida, Holcomb, Cushman and Woods2021; Woods et al., Reference Woods, Iudicello, Morgan, Cameron, Doyle, Smith and Cushman2016). As expected, we observed that neurocognitive functioning played a significant role in the observed age discrepancies in online patient portal performance in the current sample. Specifically, the primary model showed a full mediating effect of the total cognitive composite on age group differences in online patient portal performance. The effect sizes associated with the total cognitive composite mediation indicated that older adult participants were expected to obtain 1.77 points lower on TOHRN accuracy than younger adults. Importantly, the effect sizes for univariable correlations between age, the total cognitive composite and online patient portal accuracy were again quite large, suggesting that they may be of practical relevance. At face value, it is easy to appreciate the role that different aspects of cognition might play in navigating an online health portal. Once logged in, the multistep process of navigating a health portal may present a challenge for older individuals experiencing difficulties in generativity, problem-solving, and cognitive flexibility (e.g., Pak & Price, Reference Pak and Price2008). In order to follow the provider’s instructions on the TOHRN task, participants must be able read and interpret the message, disengage from the provider note, navigate a pathway to the appointment scheduling tab, and complete several sequential action steps. In this way, the observed relationship between age, online health navigation, and neurocognitive functions aligns with prior studies showing moderate associations between cognition and other aspects of internet navigation, including household tasks (e.g., shopping, banking) and health-related internet searches (Woods et al., Reference Woods, Kordovski, Tierney and Babicz2019).

Of course, any conclusions that can be drawn about the associations between age, cognition, and online health portal functioning are limited by the number and type of tests used in the battery. Given the multifactorial nature of the tests that comprised the total cognitive composite and prior studies showing domain-specificity in cognition’s role in internet navigation skills (Woods et al., Reference Woods, Kordovski, Tierney and Babicz2019), we conducted post hoc analyses to examine possible domain-level associations. Unfortunately, the tests that comprised the attention and speeded executive functions domains did not show adequate psychometrics and were dropped from further consideration. Interestingly, both of the remaining domains (i.e., verbal memory and visuomotor processing speed) mediated the association between age and TOHRN accuracy. Therefore, the current study does not provide any evidence of domain specificity in cognition’s association with online patient portal navigation. We acknowledge that absence of evidence is not evidence of absence and therefore future studies may wish to replicate these findings and include more robust domain-level measurements. Such studies might also strive to include other domains relevant to age-related cognitive declines, such as visual attention and visual memory (e.g., Gazzaley et al., Reference Gazzaley, Clapp, Kelley, McEvoy, Knight and D’Esposito2008; Madden, Reference Madden2007).

Future work is also needed to examine the possible benefits of both structural (e.g., human factors web design enhancement; Lyles et al., Reference Lyles, Adler-Milstein, Thao, Lisker, Nouri and Sarkar2021) and individual (e.g., training and compensation; Wildenbos et al., Reference Wildenbos, Jaspers and Peute2019) cognitive supports to improve the navigability of electronic patient health portals for older adults. As online health portal technology advances and adoption rates increase, overall quality of patient portals must be further investigated and improved to meet the needs of the growing number of older adult users. Older adults commonly express concern with the many challenges involved in utilization of these tools, including cumbersome processes for accessing portals, variations in provider availability for online scheduling, data security, lack of personalization, and limited technical support (e.g., Irizarry et al., Reference Irizarry, Dabbs and Curran2015). Common recommendations for increasing patient portal use among older adults and vulnerable populations have focused on screening for eHealth literacy and promoting proxy users (Price-Haywood et al., Reference Price-Haywood, Harden-Barrios, Ulep and Luo2017). In addition, developers might consider building user-friendly tutorials into the patient portal apps and sites that can help alleviate some of the cognitive demands of these tools. Examples might include a step-by-step guide of how to navigate the portal, use of larger font, the integration of salient notifications, and user-friendly, organized interfaces that could help older adults better manage the complexities of online patient portals.

Findings should be interpreted with consideration of their limitations. First and foremost, our relatively small sample poses an increased risk of Type II error, particularly as concerns the analyses of confounding factors and the generalizability of our findings. Nevertheless, we were adequately powered to detect the hypothesized primary effects. Future studies should aim to include a full lifespan sample in order to examine these effects with age as a continuous variable. Moreover, the current study included primarily college students in the younger sample and highly educated persons in the older sample, which may limit the generalizability of the findings. Second, our analyses used cross-sectional data to examine mediation, which precludes inferences regarding temporal causality between aging, neurocognitive declines, and subsequent decline in everyday online health-related behaviors. Moreover, given the simultaneous occurrence of cognitive decline and TOHRN performance inaccuracy, the relationship between age-related difficulties with neurocognition and navigating electronic patient health portals may be bidirectional. For example, older adults who have difficulty accessing and using online patient portal tools may be suboptimally engaged in their healthcare leading to poorer health status (Simmons et al., Reference Simmons, Wolever, Bechard and Snyderman2014). In turn, poorer health status may be contributing to declines in cognitive functioning (e.g., Bond et al., Reference Bond, Dickinson, Matthews, Jagger and Brayne2006). Future studies should examine whether the effect of age and online patient portal accuracy through neurocognition can be observed in prospective, longitudinal studies. Third, there are a variety of unmeasured medical and psychiatric conditions that are mild and could affect cognition that are not reflected as possible covariates in the current study (e.g., developmental disorders, pulmonary diseases).

A final word of caution is that the TOHRN is a relatively new task. Although it showed evidence of internal consistency and cross-form reliability for accuracy in this sample, its psychometric properties remain to be systematically explored (e.g., test–retest reliability, internal consistency in other populations). Although our data did not show practice effects on TOHRN accuracy and the correlation between the two versions was in the large range, the two forms of this measure cannot be considered parallel forms. Indeed, the time to completion scores were very weakly associated with one another and we are therefore unable to speak to the speeded aspects of the TOHRN. Based on the current data, future studies and clinicians should consider using only the accuracy scores from both forms in conjunction with one another. It should also be noted that the primary form of the TOHRN accuracy score shows some evidence of construct validity on its own (Woods et al., Reference Woods, Iudicello, Morgan, Cameron, Doyle, Smith and Cushman2016). Despite this caveat, the current study has several notable strengths, including the use of a performance-based internet navigation task with adequate internal consistency, the ability to manage confounds based on a well-characterized sample, and the use of clinical tests of neurocognition. Future studies may wish to examine the potential mediating effect of neurocognition on the relationship between age and a more comprehensive series of internet tasks of everyday functioning that include other measures of online healthcare (e.g., provider and insurance searches), communication (e.g., email and social networking), transportation (e.g., planning a trip on a mass transit system), and household activities (e.g., shopping and banking).

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/S1355617722000650

Acknowledgments

We are grateful to the programming team at the University of California, San Diego for their considerable efforts in developing the Test of Online Health Records Navigation (TOHRN) paradigm. We acknowledge Kishon Joseph and Nathalie Ulrich for their help with test administration and scoring for the parent project. We also thank the study participants for their time and insights.

Funding statement

This study was supported by research funds provided to the corresponding author by the University of Houston.

Conflicts of interest

None.

Footnotes

1 Recent reports suggest that older adults who use the Internet minimally report using it at least 1.6 hr per week (van Boekel et al., Reference van Boekel, Peek and Luijkx2017). Thus, the rationale for inclusion criteria of at least 1.5 hr of internet use per week was to include minimal internet use based on population internet use in order to reflect the construct we were attempting to measure (i.e., electronic patient health portal navigation).

2 We acknowledge that there is heterogeneity in how covariates can be selected. As such, we conducted post hoc analyses to confirm that the manner in which covariates were evaluated did not meaningfully affect the final results. The observed mediating effects of the total cognitive composite and Domain-Level cognition were not altered when we instead used either a true covariate approach (i.e., including any variable that differed significantly by age group irrespective of their relationship to TOHRN) or an a priori approach (i.e., medical comorbidities and internet use). In other words, the mediating effects of the total cognitive composite and Domain-Level cognition remained consistent with multiple approaches to determining covariates.

3 We chose to report on accuracy scores instead of time to completion to structurally limit the risk of Type 1 error, and out of presumed lack of general community interest about showing age effects on another speeded test. Nevertheless, we do see an effect of age on TOHRN time to completion (b = 160.3, 95% CI [54.1, 266.5), which is partially mediated by the total cognitive Z-score PCA composite (b = 0.36, 95% CI [0.15, 0.57]).

References

Agree, E. M., King, A. C., Castro, C. M., Wiley, A., & Borzekowski, D. L. (2015). “It’s got to be on this page”: Age and cognitive style in a study of online health information seeking. Journal of Medical Internet Research, 17, e79. https://doi.org/10.2196/jmir.3352 CrossRefGoogle Scholar
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing, Inc. https://doi.org/10.1176/appi.books.9780890425596 Google Scholar
Baggio, S., Iglesias, K., Berchtold, A., & Suris, J.-C. (2017). Measuring Internet use: Comparisons of different assessments and with Internet addiction. Addiction Research & Theory, 25, 114120. https://doi.org/10.1080/16066359.2016.1206083 CrossRefGoogle Scholar
Bond, J., Dickinson, H. O., Matthews, F., Jagger, C., & Brayne, C. (2006). Self-rated health status as a predictor of death, functional and cognitive impairment: A longitudinal cohort study. European Journal of Ageing, 3, 193206. https://doi.org/10.1007/s10433-006-0039-8 CrossRefGoogle ScholarPubMed
Chevalier, A., Dommes, A., & Marquié, J.-C. (2015). Strategy and accuracy during information search on the Web: Effects of age and complexity of the search questions. Computers in Human Behavior, 53, 305315. https://doi.org/10.1016/j.chb.2015.07.017 CrossRefGoogle Scholar
Choi, N. G., & DiNitto, D. M. (2013). The digital divide among low-income homebound older adults: Internet use patterns, eHealth literacy, and attitudes toward computer/Internet use. Journal of Medical Internet Research, 15, e93. https://doi.org/10.2196/jmir.2645 CrossRefGoogle ScholarPubMed
Coughlin, S. S., Stewart, J. L., Young, L., Heboyan, V., & De Leo, G. (2018). Health literacy and patient web portals. International Journal of Medical Informatics, 113, 4348. https://doi.org/10.1016/j.ijmedinf.2018.02.009 CrossRefGoogle ScholarPubMed
Czaja, S. J., Sharit, J., Ownby, R., Roth, D. L., & Nair, S. (2001). Examining age differences in performance of a complex information search and retrieval task. Psychology and Aging, 16, 564579. http://doi.org/10.1037/0882-7974.16.4.564 CrossRefGoogle ScholarPubMed
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 319340. https://doi.org/10.2307/249008 CrossRefGoogle Scholar
Delis, D. C., Kaplan, E., & Kramer, J. H. (2001). Delis-Kaplan executive function system (D-KEFS). Psychological Corporation. https://doi.org/10.1037/t15082-000 Google Scholar
Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (2000). CVLT-II California verbal learning test: adult version. Psychological Corporation.Google Scholar
Department of Health and Human Services. (2017). What is a patient portal? HealthIT.gov. https://www.healthit.gov/faq/what-patient-portal Google Scholar
Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. Sage publications.Google Scholar
Field-Fote, E. (2019). Mediators and moderators, confounders and covariates: Exploring the variables that illuminate or obscure the “active ingredients” in neurorehabilitation. Journal of Neurologic Physical Therapy, 43, 8384.CrossRefGoogle Scholar
Floyd, F. J., & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286. https://doi.org/10.1037/1040-3590.7.3.286 CrossRefGoogle Scholar
Fritz, M. S., & MacKinnon, D. P. (2008). A graphical representation of the mediated effect. Behavior Research Methods, 40, 5560. https://doi.org/10.3758/BRM.40.1.55 CrossRefGoogle ScholarPubMed
Gazzaley, A., Clapp, W., Kelley, J., McEvoy, K., Knight, R. T., & D’Esposito, M. (2008). Age-related top-down suppression deficit in the early stages of cortical visual memory processing. Proceedings of the National Academy of Sciences, 105, 1312213126.CrossRefGoogle ScholarPubMed
Glorfeld, L. W. (1995). An improvement on Horn’s parallel analysis methodology for selecting the correct number of factors to retain. Educational and Psychological Measurement, 55, 377393. https://doi.org/10.1177/0013164495055003002 CrossRefGoogle Scholar
Han, H. R., Gleason, K. T., Sun, C. A., Miller, H. N., Kang, S. J., Chow, S., Anderson, R., Nagy, P., & Bauer, T. (2019). Using patient portals to improve patient outcomes: Systematic review. JMIR Human Factors, 6, e15038. https://doi.org/10.2196/15038 CrossRefGoogle ScholarPubMed
Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: a regression-based approach. Guilford Press. http://www.jstor.org/stable/24018134 Google Scholar
Henry, J., Barker, W., & Kachay, L. (2019). Electronic capabilities for patient engagement among U.S. non-federal acute care hospitals: 2013–2017. ONC data brief. Office of the National Coordinator for Health Information Technology.Google Scholar
Hong, Y. A., Jiang, S., & Liu, P. L. (2020). Use of patient portals of electronic health records remains low from 2014 to 2018: Results from a national survey and policy implications. American Journal of Health Promotion, 34, 677680. https://doi.org/10.1177/0890117119900591 CrossRefGoogle ScholarPubMed
Irizarry, T., Dabbs, A. D., & Curran, C. R. (2015). Patient portals and patient engagement: A state of the science review. Journal of Medical Internet Research, 17, e148. https://doi.org/10.2196/jmir.4255 CrossRefGoogle ScholarPubMed
Joiner, R., Brosnan, M., Duffield, J., Gavin, J., & Maras, P. (2007). The relationship between Internet identification, Internet anxiety and Internet use. Computers in Human Behavior, 23, 14081420. https://doi.org/10.1016/j.chb.2005.03.002 CrossRefGoogle Scholar
Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141151. https://doi.org/10.1177/001316446002000116 CrossRefGoogle Scholar
Kordovski, V. M., Tierney, S. M., Rahman, S., Medina, L. D., Babicz, M.A., Yoshida, H., Holcomb, E. M., Cushman, C., & Woods, S.P. (2021). Older age and online health information search behaviors: The mediating influence of executive functions. Journal of Clinical and Experimental Neuropsychology, 43, 689703. https://doi.org/10.1080/13803395.2021.1990866 CrossRefGoogle ScholarPubMed
Lipkus, I. M., Samsa, G., & Rimer, B. K. (2001). General performance on a numeracy scale among highly educated samples. Medical Decision Making, 21, 3744. https://doi.org/10.1177/0272989X0102100105 CrossRefGoogle ScholarPubMed
Lyles, C. R., Adler-Milstein, J., Thao, C., Lisker, S., Nouri, S., & Sarkar, U. (2021). Alignment of key stakeholders’ priorities for patient-facing tools in digital health: Mixed methods study. Journal of Medical Internet Research, 23, e24890. https://doi.org/10.2196/24890 CrossRefGoogle ScholarPubMed
MacKinnon, D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation analysis. Annual Review of Psychology, 58, 593614. https://doi.org/10.1146/annurev.psych.58.110405.085542 CrossRefGoogle ScholarPubMed
Madden, D. J. (2007). Aging and visual attention. Current Directions in Psychological Science, 16, 7074. https://doi.org/10.1111/j.1467-8721.2007.00478.x CrossRefGoogle ScholarPubMed
Manning, R. T. (1982). The serial sevens test. Archives of Internal Medicine, 142, 11921192.CrossRefGoogle ScholarPubMed
Nahm, E. S., Zhu, S., Bellantoni, M., Keldsen, L., Charters, K., Russomanno, V., Rietschel, M., Son, H., & Smith, L. (2020). Patient portal use among older adults: What is really happening nationwide? Journal of Applied Gerontology, 39, 442450.CrossRefGoogle ScholarPubMed
Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., Cummings, J. L., & Chertkow, H. (2005). The Montreal cognitive assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53, 695699. https://doi.org/10.1111/j.1532-5415.2005.53221.x CrossRefGoogle Scholar
Norman, C. D., & Skinner, H. A. (2006). eHEALS: The eHealth literacy scale. Journal of Medical Internet Research, 8, e27. https://doi.org/10.2196/jmir.8.4.e27 CrossRefGoogle ScholarPubMed
O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32, 396402. https://doi.org/10.3758/BF03200807 CrossRefGoogle ScholarPubMed
Pak, R., & Price, M. M. (2008). Designing an information search interface for younger and older adults. Human Factors, 50, 614628. https://doi.org/10.1518/001872008X312314 CrossRefGoogle ScholarPubMed
Park, D. C. (1992). Applied cognitive aging research. In Craik, F. I. M. & Salthouse, T. A. (Eds.), The handbook of aging and cognition (pp. 449493). Lawrence Erlbaum Associates.Google Scholar
Piatt, A. L., Fields, J. A., Paolo, A. M., Koller, W. C., & Tröster, A. I. (1999). Lexical, semantic, and action verbal fluency in Parkinson’s disease with and without dementia. Journal of Clinical and Experimental Neuropsychology, 21, 435443. https://doi.org/10.1076/jcen.21.4.435.885 CrossRefGoogle ScholarPubMed
Price-Haywood, E. G., Harden-Barrios, J., Ulep, R., & Luo, Q. (2017). eHealth literacy: Patient engagement in identifying strategies to encourage use of patient portals among older adults. Population Health Management, 20, 486494. https://doi.org/10.1089/pop.2016.0164 CrossRefGoogle ScholarPubMed
Rahman, S., Kordovski, V. M., Tierney, S. M., & Woods, S. P. (2021). Internet navigation skills for financial management: Associations with age, neurocognitive performance, and functional capacity. Neuropsychology, 35, 630642. https://doi.org/10.1037/neu0000759 CrossRefGoogle ScholarPubMed
Rossetti, H. C., Lacritz, L. H., Cullum, C. M., & Weiner, M. F. (2011). Normative data for the Montreal cognitive assessment (MoCA) in a population-based sample. Neurology, 77, 12721275. https://doi.org/10.1212/WNL.0b013e318230208a CrossRefGoogle Scholar
Sakaguchi-Tang, D. K., Bosold, A. L., Choi, Y. K., & Turner, A. M. (2017). Patient portal use and experience among older adults: Systematic review. JMIR Medical Informatics, 5, e38. https://doi.org/10.2196/medinform.8092 CrossRefGoogle ScholarPubMed
Sangha, O., Stucki, G., Liang, M. H., Fossel, A. H., & Katz, J. N. (2003). The self-administered comorbidity questionnaire: A new method to assess comorbidity for clinical and health services research. Arthritis Care & Research, 49, 156163. https://doi.org/10.1002/art.10993 CrossRefGoogle ScholarPubMed
Schroeder, R. W., Twumasi-Ankrah, P., Baade, L. E., & Marshall, P. S. (2012). Reliable digit span: A systematic review and cross-validation study. Assessment, 19, 2130. https://doi.org/10.1177/1073191111428764 CrossRefGoogle Scholar
Simmons, L. A., Wolever, R. Q., Bechard, E. M., & Snyderman, R. (2014). Patient engagement as a risk factor in personalized health care: A systematic review of the literature on chronic disease. Genome Medicine, 6, 113. https://doi.org/10.1186/gm533 CrossRefGoogle ScholarPubMed
Small, G. W., Moody, T. D., Siddarth, P., & Bookheimer, S. Y. (2009). Your brain on Google: Patterns of cerebral activation during internet searching. The American Journal of Geriatric Psychiatry, 17, 116126. https://doi.org/10.1097/JGP.0b013e3181953a02 CrossRefGoogle ScholarPubMed
Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Pearson Education.Google Scholar
Tierney, S. M., Kordovski, V. M., Rahman, S., Medina, L. D., Damian, R. I., Collins, R. L., & Woods, S. P. (2022). Neuropsychological aspects of Internet-based transit navigation skills in older adults. Aging, Neuropsychology and Cognition, 29, 87103. https://doi.org/10.1080/13825585.2020.1852164 CrossRefGoogle ScholarPubMed
Tucker-Drob, E. M. (2011). Neurocognitive functions and everyday functions change together in old age. Neuropsychology, 25, 368377. https://doi.org/10.1037/a0022348 CrossRefGoogle ScholarPubMed
van Boekel, L. C., Peek, S. T., & Luijkx, K. G. (2017). Diversity in older adults’ use of the internet: Identifying subgroups through latent class analysis. Journal of Medical Internet Research, 19, e180. https://doi.org/10.2196/jmir.6853 CrossRefGoogle ScholarPubMed
Van den Broeck, J., Argeseanu Cunningham, S., Eeckels, R., & Herbst, K. (2005). Data cleaning: Detecting, diagnosing, and editing data abnormalities. PLoS Medicine, 2, e267.CrossRefGoogle ScholarPubMed
Wechsler, D. (2008). Wechsler adult intelligence scale – (WAIS-IV) (4th ed.). NCS Pearson.Google Scholar
Wecker, N. S., Kramer, J. H., Wisniewski, A., Delis, D. C., & Kaplan, E. (2000). Age effects on executive ability. Neuropsychology, 14, 409414. https://doi.org/10.1037/0894-4105.14.3.409 CrossRefGoogle ScholarPubMed
Weiss, B. D., Mays, M. Z., Martz, W., Castro, K. M., DeWalt, D. A., Pignone, M. P., Mockbee, J., & Hale, F. A. (2005). Quick assessment of literacy in primary care: The newest vital sign. Annals of Family Medicine, 3, 514522. https://doi.org/10.1370/afm.405 CrossRefGoogle ScholarPubMed
Wildenbos, G. A., Jaspers, M., & Peute, L. (2019). The equity paradox: Older patients’ participation in patient portal development. International Journal for Quality in Health Care, 31, 793797. https://doi.org/10.1093/intqhc/mzy245 Google ScholarPubMed
Wildenbos, G. A., Maasri, K., Jaspers, M., & Peute, L. (2018). Older adults using a patient portal: Registration and experiences, 1 year after implementation. Digital Health, 4, 2055207618797883. https://doi.org/10.1177/2055207618797883 CrossRefGoogle Scholar
Wilkinson, G. S., & Robertson, G. J. (2006). Wide range achievement test 4 (WRAT4). Psychological Assessment Resources.Google Scholar
Woods, S. P., Iudicello, J. E., Morgan, E. E., Cameron, M. V., Doyle, K. L., Smith, T. V., Cushman, C., & The HNRP Group. (2016). Health-related everyday functioning in the Internet age: HIV-associated neurocognitive disorders disrupt online pharmacy and health chart navigation skills. Archives of Clinical Neuropsychology, 31, 176185. http://doi.org/10.1093/arclin/acv090 Google ScholarPubMed
Woods, S. P., Kordovski, V. M., Tierney, S. M., & Babicz, M. A. (2019). The neuropsychological aspects of performance-based Internet navigation skills: A brief review of an emerging literature. The Clinical Neuropsychologist, 33, 305326. https://doi.org/10.1080/13854046.2018.1503332 CrossRefGoogle ScholarPubMed
Woods, S. P., Scott, J. C., Sires, D. A., Grant, I., Heaton, R. K., Tröster, A. I., & The HNRC Group. (2005). Action (verb) fluency: Test-retest reliability, normative standards, and construct validity. Journal of the International Neuropsychological Society, 11, 408415. https://doi.org/10.1017/S1355617705050460 CrossRefGoogle ScholarPubMed
Yong, A. G., & Pearce, S. (2013). A beginner’s guide to factor analysis: Focusing on exploratory factor analysis. Tutorials in Quantitative Methods for Psychology, 9, 7994. https://doi.org/10.20982/tqmp.09.2.p079 CrossRefGoogle Scholar
Figure 0

Table 1. Descriptive characteristics of the older and younger study groups

Figure 1

Table 2. Total cognitive composite principal component analysis (PCA) loadings

Figure 2

Table 3. Zero-order correlation matrix of all primary variables in the mediation models

Figure 3

Figure 1. Mediation model with partially standardized bootstrap estimates and 95% confidence interval values for age group (n = 35 older adults; n = 49 younger adults) and TOHRN accuracy mediated by the total cognitive composite.

Supplementary material: File

Matchanova et al. supplementary material

Tables S1-S3

Download Matchanova et al. supplementary material(File)
File 56.1 KB