Hostname: page-component-586b7cd67f-l7hp2 Total loading time: 0 Render date: 2024-11-27T04:47:44.258Z Has data issue: false hasContentIssue false

2 Validity and Reliability of Mobile Toolbox Cognitive Assessments

Published online by Cambridge University Press:  21 December 2023

Cindy J Nowinski*
Affiliation:
Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Aaron Kaat
Affiliation:
Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Jerry Slotkin
Affiliation:
University of Delaware, Newark, Del, USA
Erika La Forte
Affiliation:
Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Yusuke Shono
Affiliation:
Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Miriam Novack
Affiliation:
Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Sarah Pila
Affiliation:
Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Elizabeth Dworak
Affiliation:
Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Stephanie R Young
Affiliation:
Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Zahra Hosseinian
Affiliation:
Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Hubert Adam
Affiliation:
Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Richard Gershon
Affiliation:
Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
*
Correspondence: Cindy J. Nowinski, Northwestern University Feinberg School of Medicine
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.
Objective:

To present validation evidence for the first eight cognitive measures available through Mobile Toolbox (MTB). These measures use a remote self-administered platform to assess language, working memory, episodic memory, executive function, and processing speed.

Participants and Methods:

We used two separate samples, recruited as part of a larger study, to validate MTB measures. Sample I, comprised of 92 English-speaking adults ages 18-85, was used to assess internal consistency and construct validity. Participants were first administered “gold standard” cognitive measures (Wechsler Memory Scale-IV Verbal Paired Associates I and II; Wechsler Adult Intelligence Scale-IV Symbol Search, Digit Span, Coding, and Letter-Number Sequencing; Delis-Kaplan Executive Function System Color-Word Interference Test, Peabody Picture Vocabulary Test, Wechsler Individual Achievement Test-4 Spelling, and the Wisconsin Card Sorting Test), after which they completed MTB (pre-loaded on a study-provided smartphone) on their own. Internal consistency was evaluated using measure-appropriate indices (split-half reliability, Cronbach’s alpha or IRT-based indices). Pearson correlation coefficients between MTB tests and measures of similar constructs were used to evaluate concurrent validity. For two tests with timing-dependent scores, Arrow Matching and Shape-Color Sorting, separate analyses were performed for iOS and Android devices. Sample II, with 1,120 English-speaking participants ages 18-90, was used to evaluate age-related change. Participants completed MTB measures remotely on their own smartphones, in a preset order, within a 14-day period. Spearman correlation coefficients, corrected for education, were calculated to evaluate relationships between age and test scores.

Results:

Sample I participants were 67% female, 52% white, 99% non-Hispanic; average age=48 (SD= 17). Education was: < high school (1%); high school (55%); some college (21%); college (15%); graduate degree (8%). Internal consistency estimates ranged from 0.81 to 0.99. Pearson correlations between MTB and external measures ranged from 0.41 to 0.86 (all p < .01). Of the timed tests, only Shape-Color sorting showed significant score differences between Android and iOS devices. Sample II was 57% female, 13% Hispanic, 72% white, mean age = 45 (SD = 21). Education distribution was: < high school (2%); high school (34%); some college (34%), college (20%); graduate degree (11%). Measures of executive function (r = -0.50; r=-0.57) and processing speed (r= -0.61) showed the expected negative correlation with age (all p <0.001). Negative correlations, although weaker, were also seen on measures of working memory (r=-0.2) and episodic memory (r=-0.2, r=-0.37; p.<.001). Vocabulary performance improved with age (r=0.4; p<.001), while spelling scores remained stable (r=0.09).

Conclusions:

Initial studies support the validity and reliability of the first eight MTB cognitive measures in two diverse samples. MTB tests showed satisfactory construct validity, as demonstrated by the associations between MTB and well-established tests. Furthermore, most MTB measures correlated with age in the expected directions. Executive function, processing speed and memory typically decrease with age and this decrease was reflected in MTB test performance. In contrast, spelling and vocabulary, typically preserved as we age, did not decrease in our sample. Our results support the use of MTB in cognitive aging research.

Type
Poster Session 08: Assessment | Psychometrics | Noncredible Presentations | Forensic
Copyright
Copyright © INS. Published by Cambridge University Press, 2023