Hostname: page-component-586b7cd67f-t7czq Total loading time: 0 Render date: 2024-11-24T11:22:33.980Z Has data issue: false hasContentIssue false

Do normative data specific to Greek Australian older adults improve validity of neuropsychological assessment results?

Published online by Cambridge University Press:  22 November 2023

Mathew Staios*
Affiliation:
Turner Institute for Brain and Mental Health, Monash University, Melbourne, Australia Monash-Epworth Rehabilitation Research Centre, Melbourne, Australia
Mary H. Kosmidis
Affiliation:
Lab of Cognitive Neuroscience, School of Psychology, Aristotle University of Thessaloniki, Thessaloniki, Greece
Yiannis Tsiaras
Affiliation:
Lab of Cognitive Neuroscience, School of Psychology, Aristotle University of Thessaloniki, Thessaloniki, Greece
T. Rune Nielsen
Affiliation:
Department of Neurology, Neuroscience Centre, Danish Dementia Research Centre, Copenhagen University Hospital, Copenhagen, Denmark
Alexandra Papadopoulos
Affiliation:
Turner Institute for Brain and Mental Health, Monash University, Melbourne, Australia
Arthur Kokkinias
Affiliation:
The Royal Melbourne Hospital, Inner West Area Mental Health Service, Melbourne, Australia
Dennis Velakoulis
Affiliation:
Department of Psychiatry, Melbourne Neuropsychiatry Centre, The University of Melbourne and Melbourne Health, Melbourne, Australia Neuropsychiatry, The Royal Melbourne Hospital, Parkville, Melbourne, Australia
Evrim March
Affiliation:
Turner Institute for Brain and Mental Health, Monash University, Melbourne, Australia
Renerus J. Stolwyk*
Affiliation:
Turner Institute for Brain and Mental Health, Monash University, Melbourne, Australia Monash-Epworth Rehabilitation Research Centre, Melbourne, Australia
*
Corresponding authors: R. J. Stolwyk; Email: [email protected], M. Staios; Email: [email protected]
Corresponding authors: R. J. Stolwyk; Email: [email protected], M. Staios; Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Objective:

This study aimed to compare Greek Australian and English language normative data with regard to impairment rates yielded within a healthy Greek Australian older adult sample. We also examined whether optimal cut scores could be identified and capable of sensitively and specifically distinguishing between healthy Greek Australians from those with a diagnosis of Alzheimer’s disease (AD).

Method:

Ninety healthy Greek Australian older adults and 20 demographically matched individuals with a diagnosis of AD completed a range of neuropsychological measures, including the Wechsler Adult Intelligence Scale-Fourth Edition, Greek Adaptation (WAIS-IV GR), verbal and visual memory, language and naming, and executive functions. Impairment rates derived from the use of either Greek Australian or English language normative data were calculated and compared, using a 1.5 standard deviation criterion to denote impairment. Receiver operating characteristics curve analysis was used to investigate the sensitivity and specificity of alternate cut scores.

Results:

Impairment rates derived from the Greek Australian normative data showed that rates of impairment generally fell within the expected 7% range. In contrast, impairment rates for all tests derived using English language normative data were significantly higher and ranged from 11%–66%. Comparisons between healthy and AD participants with moderate dementia showed significant differences across all measures. Area under the curve results ranged from .721 to .999 across all measures, with most tests displaying excellent sensitivity and specificity.

Conclusions:

English language normative data were found to be inappropriate for use with Greek Australian elders, potentially leading to erroneous diagnostic outcomes. The use of minority group specific normative data and associated cut points appear to partially ameliorate this issue. Clinical implications are discussed alongside future research directions.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © INS. Published by Cambridge University Press 2023

As of 2020, there were over 55 million people worldwide living with dementia. This number is expected to almost double every 20 years, reaching approximately 139 million by 2050 (World Health Organisation, 2022). Projected trends suggest that over 700,000 Australian older adults are expected to meet criteria for dementia by 2050, with one-third consisting of individuals from culturally and linguistically diverse backgrounds (Australian Institute of Health & Welfare, 2018). Research has consistently reported that many English language neuropsychological measures and normative data are prone to misclassification when applied to culturally and linguistically diverse groups (Daugherty et al., Reference Daugherty, Puente, Fasfous, Hidalgo-Ruzzante and Pérez-Garcia2017; Heaton et al., Reference Heaton, Taylor, Manly, Tulsky, Saklofske, Chelune, Heaton, Ivnik, Bornstein, Prifitera and Ledbetter2003). Poor test specificity has been attributed to cultural and linguistic heterogeneity, level and quality of education, use of unrepresentative normative data, culturally biased test content, and test-taking attitudes (Rivera Mindt et al., Reference Rivera Mindt, Byrd, Saez and Manly2010; Shuttleworth-Edwards, Reference Shuttleworth-Edwards2016). These issues have been extensively examined within minority groups in the United States, including African Americans and Hispanics, where healthy individuals were over three times more likely to be misclassified as impaired (Heaton et al., Reference Heaton, Taylor, Manly, Tulsky, Saklofske, Chelune, Heaton, Ivnik, Bornstein, Prifitera and Ledbetter2003). In response to this issue, demographically focused normative data have been developed across a range of culturally diverse populations (Nielsen et al., Reference Nielsen, Segers, Vanderaspoilden, Bekkhus-Wetterberg, Minthon, Pissiota, Bjørkløf, Beinhoff, Tsolaki, Gkioka and Waldemar2018; Pienaar et al., Reference Pienaar, Shuttleworth-Edwards, Klopper and Radloff2016). While it is anticipated that normative data specifically developed for ethnic minority groups will improve the validity of neuropsychological assessment, surprisingly limited empirical research has examined this issue. The Neuropsychological Norms for the U.S.-Mexico Border Region in Spanish Project (NP-NUMBRS) reported that their adapted tests and normative data more accurately classified healthy Spanish-speaking individuals and yielded expected rates of impairment for a majority of measures compared to published normative data for English-speaking non-Hispanic Whites and Blacks in the United States (Díaz-Santos et al., Reference Díaz-Santos, Suárez, Marquine, Umlauf, Rivera Mindt, Fortuny, Heaton, R. and Cherner2021; Marquine et al., Reference Marquine, Morlett Paredes, Madriaga, Blumstein, Umlauf, Kamalyan and Cherner2020; Marquine et al., Reference Marquine, Yassai-Gonzalez, Perez-Tejada, Umlauf, Kamalyan, Morlett Paredes, Rivera Mindt, Artiola, Fortuny, Cherner and Heaton2021; Scott et al., Reference Scott, Morlett Paredes, Taylor, Umlauf, Fortuny, Heaton, R., Cherner, Marquine and Rivera Mindt2021; Suarez et al., Reference Suarez, Díaz-Santos, Marquine, Kamalyan, Mindt, Umlauf, Heaton, Grant and Cherner2021). However, further research across multiple international contexts is required to ascertain whether the use of adapted tests and minority group reference normative data indeed results in reducing misclassification rates in healthy culturally diverse individuals and improves diagnostic accuracy across a range of neurocognitive disorders.

In the absence of valid and reliable neuropsychological measures, accurate assessment of cognitive disorders within ethnic minority and/or non-English-speaking groups continues to be an ongoing challenge facing the global neuropsychological community, including Australia (Blakemore et al., Reference Blakemore, Kenning, Mirza, Daker-White, Panagioti and Waheed2018; Franzen et al., Reference Franzen, Papma, van den Berg and Nielsen2021; Rivera Mindt et al., Reference Rivera Mindt, Byrd, Saez and Manly2010; Wallace et al., Reference Wallace, Berry and Shores2018). Similar to other developed nations, illiteracy rates in Australia are very low and the average level of education is high (Australian Institute of Health and Welfare, 2021). In contrast, a large proportion of aging Australian ethnic minorities attained low levels and a poorer quality education prior to immigrating, relative to standards noted in their Australian-born elderly counterparts (Fratti et al., Reference Fratti, Bowden and Pino2011; Jupp, Reference Jupp2001; Noutsos, Reference Noutsos2003; Plitas et al., Reference Plitas, Tucker, Kritikos, Walters and Bardenhagen2009). A case in point is elderly Greek Australian immigrants, who represent one of the largest aging ethnic minority groups in Australia, totaling approximately 63,000 individuals aged 65 years and older, with over 40,000 having attained a primary level of education or less (Australian Bureau of Statistic, 2016; Fanany & Avgoulas, Reference Fanany, Avgoulas and Ratuva2019; Staios, Reference Staios and Irani2022). Accordingly, performance on most neuropsychological tests is heavily influenced by years (and quality) of education (Shuttleworth-Edwards, Reference Shuttleworth-Edwards2016). However, older low educated immigrant groups are underrepresented in a majority of normative studies, making it extremely difficult to interpret what constitutes a normal performance on several neuropsychological measures (Kosmidis, Reference Kosmidis2017; Nielsen & Waldemar, Reference Nielsen and Waldemar2021). Given the noted limitations with using English language tests and normative data, we adapted test content and developed normative data for a broad range of neuropsychological measures, including general intelligence (i.e., Wechsler Adult Intelligence Scale-Fourth Edition [WAIS-IV] Greek Adaptation; Wechsler, Reference Wechsler2014), verbal and visual memory tests, language and naming, and executive functioning measures for use with Greek Australian older adults (Staios et al., Reference Staios, Kosmidis, Kokkinis, Papadopoulos, Nielsen, Kalinowski, March and Stolwyk2023a, b). The first aim of this study was to compare our newly established Greek Australian normative data with existing published English language normative data in terms of impairment rates they yield within a healthy Greek Australian older adult sample. We hypothesized that using Greek Australian normative data would produce lower impairment rates, while English language normative data would produce significantly higher rates of impairment. The second aim of the study was to examine whether Greek Australian normative data could yield sensitive and specific cut scores to distinguish healthy Greek Australians from Greek Australians with a diagnosis of Alzheimer’s disease (AD).

Method

Participants

Healthy participants were recruited using convenience sampling from several Greek social clubs throughout the Melbourne metropolitan area. In order to be eligible for participation, participants had to be aged between 70–85 years, literate, and immigrants from Greece, with Greek as their dominant language. The exclusion criteria for the participants were: (i) a score on the Mini-Mental State Examination below 22/30 (MMSE; Fountoulakis et al., Reference Fountoulakis, Tsolaki, Chantzi and Kazis2000; Plitas et al., Reference Plitas, Tucker, Kritikos, Walters and Bardenhagen2009); (ii) a score of < 0.5 on the Clinical Dementia Rating Scale (CDR; Hughes et al., Reference Hughes, Berg, Danziger, Coben and Martin1982); (iii) history of serious neurological or psychiatric conditions known to impact cognition; (iv) a score ≥ 6/15 on the Geriatric Depression Scale-15 (GDS-15) (Fountoulakis et al., Reference Fountoulakis, Tsolaki, Iacovides, Yesavage, O’Hara and Kazis1999) or a score of ≥ 8/20 on the Geriatric Anxiety Inventory (GAI; Pachana et al., Reference Pachana, Byrne, Siddle, Koloski, Harley and Arnold2007); (v) evidence of long term alcohol or substance abuse; and (vi) uncorrected sensory and/or motor deficits.

Participants with AD were recruited via specialist outpatient clinics throughout the Melbourne metropolitan area. Inclusion and exclusion criteria were the same as for healthy participants but also requiring a probable diagnosis of AD (therefore MMSE, CDR exclusion criteria did not apply). The clinical diagnosis of AD was classified using the following methods: interview with the patient and (when possible) an informant; a neurological, physical, and psychiatric examination; cognitive screening with the Greek version of the MMSE or the Neuropsychiatry Unit Cognitive Assessment Tool (NUCOG; Walterfang et al., Reference Walterfang, Velakoulis, Gibbs and Lloyd2003); a score of ≥1 on the CDR; laboratory screening with blood tests, and structural brain imaging with computerized tomography, magnetic resonance imaging, and/or functional brain imaging with single-photon emission computed tomography. Diagnoses were based on using evidence from all clinical results, in conjunction with the established diagnostic criteria as described by the Diagnostic and Statistical Manual of Mental Disorders—Fifth Edition (DSM-5; American Psychiatric Association, 2013) for neurocognitive disorders and the diagnostic research criteria for AD (Dubois et al., Reference Dubois, Feldman, Jacova, DeKosky, Barberger-Gateau and Cummings2007).

Measures

The measures presented in the section to follow were previously administered in the Greek language and normed for use with Greek Australian older adults. This has been described in detail elsewhere (Staios et al., Reference Staios, Kosmidis, Kokkinis, Papadopoulos, Nielsen, Kalinowski, March and Stolwyk2023a, b). To compare performances between normative data sets, we selected comparable Australian tests and normative data routinely used in clinical practice for the assessment of older adults. In the absence of local Australian normative studies, we employed tests and normative data from the United States and Canada, which are also routinely used for the assessment of older adults in Australia. For the present study, the term English language normative data will be used throughout this paper when referring to existing published normative data developed for the majority English-speaking population, including normative data published in test manuals and research papers.

Cognitive screener

Mini-Mental State Exam – Greek Adaptation (MMSE; Fountoulakis et al., Reference Fountoulakis, Tsolaki, Chantzi and Kazis2000). Previous research has established that using the recommended Greek national MMSE cutoff of < 24/30 has resulted in large proportions of healthy Greek Australian older adults falling within the impaired range (i.e., 35% scored below the MMSE cutoff score of < 24/30; Plitas et al., Reference Plitas, Tucker, Kritikos, Walters and Bardenhagen2009). Therefore, we selected an MMSE cutoff of < 22/30, which has been recommended when screening low educated ethnic minority populations (Kochhann et al., Reference Kochhann, Varela, de Macedo Lisboa and Chaves2010; Pedraza et al., Reference Pedraza, Clark, O’Bryant, Smith, Ivnik, Graff-Radford, Lucas, Willis, Petersen and Lucas2011).

General intelligence

The Wechsler Adult Intelligence Scale-Fourth Edition, Greek Adaptation (WAIS-IV GR; Wechsler, Reference Wechsler2014). The Greek version of the WAIS-IV was adapted from the original version of the WAIS-IV (Wechsler, Reference Wechsler2008) and includes the same subtests, items, and score structure with only minor cultural adaptations in item content/scoring. It includes 10 core and five supplemental subtests and yields a full-scale IQ (FSIQ) and four index scores (verbal comprehension index, VCI; perceptual reasoning index, PRI; working memory index, WMI, and processing speed index, PSI). English language normative data used to calculate standard performances were derived using US data (Wechsler, Reference Wechsler2008).

New learning & memory

The Hopkins Verbal Learning Test – Revised Form 5 (HVLT-R; Brandt & Benedict, Reference Brandt and Benedict2001). The HVLT-R is a verbal memory task consisting of 12 words from three semantic categories (professions, food, and sports) presented over three trials, with correct words summed to equal a total score (range = 0–36). Following a 30-min interference interval, participants are required to recall as many words learned from the list presented (range = 0–12), followed by a yes/no recognition trial. The recognition component has a total of 24 words: 12 target words, 6 semantically related words, and 6 semantically unrelated words. English language normative data used to calculate standard performances were derived using Australian data (Hester et al., Reference Hester, Kinsella, Ong and Turner2004).

Greek Story Memory Tests (Kosmidis et al., Reference Kosmidis, Bozikas and Vlahou2012). The Greek Story Memory Tests assesses narrative memory under a free recall condition. A short story relating to a couple taking a holiday on a Greek island is presented over two trials, with immediate recall after each trial (range = 0–32). After a 30-min interference interval, participants are required to recall as much of the story as possible (range = 0–16), followed by a 10-question recognition trial (range = 0–10).

Wechsler Memory Scale - Fourth Addition (WMS-IV; Wechsler, Reference Wechsler2009) Visual Reproduction. WMS-IV Visual Reproduction assesses memory for visual stimuli. Five pages of designs are presented in turn, for 10 s each. After each page is presented, the examinee is asked to draw the designs (range = 0–43). After a 30-min interference interval, participants are asked to draw as many designs as they can remember, in any order (range = 0–43). Finally, a recognition trial is administered where the examinee is required to select the target design among six options (range = 0–7). English language normative data used to calculate standard performances were derived using US data (Wechsler, Reference Wechsler2009).

Language & naming

Greek Naming Test (Kosmidis et al., Reference Kosmidis, Bozikas and Vlahou2012). The Greek Naming Test requires participants to name a series of 40 black and white pictures of common objects, giving a range of scores from 0 to 40 points.

Executive functions

Color Trails Test (CTT; D’Elia et al., Reference D’Elia, Satz, Uchiyama and White1996). The CTT is a nonalphabetical version of the Trail Making Test (Reitan, Reference Reitan1955) that was developed for cross-cultural populations. In CTT 1, participants are required to connect numbered circles in ascending order. In CTT 2, participants are required to switch between pink and yellow colors while connecting numbers in an ascending order (i.e., pink 1, yellow 2, pink 3, yellow 4, and so on). We recorded the time for the completion for both CTT 1 and 2 in seconds. We also recorded number errors, color errors, self-corrected errors, prompts, and the interference index (ratio of CTT 2 – 1/CTT 1 time scores). English language normative data used to calculate standard performances were derived using US data (D’Elia et al., Reference D’Elia, Satz, Uchiyama and White1996).

Semantic Verbal Fluency (Kosmidis et al., Reference Kosmidis, Vlahou, Panagiotaki and Kiosseoglou2004). Participants were required to generate as many different animal names and ‘things you can buy in a supermarket’ within 1 min. English language normative data used to calculate standard performances were derived using Australian data (Wardill et al., Reference Wardill, Anderson, Graham and Perre2009).

Victoria Stoop Test (Regard, Reference Regard1981): The Victoria Stoop Test consists of three A4 cards, each containing six rows of four items. In Part D (Dots), participants must name as quickly as possible the color of 24 dots printed in blue, green, red, or yellow. In Part W (Words) the dots are replaced by common words (when, hard, and over), where participants are required to name the colors of the words printed. In Part C (Colors) stimuli are the color names (blue, green, red, and yellow) printed in a color that does not correspond to the name of the color (e.g., “red” is written in blue ink). For each condition, the scores are the time of completion and the number of uncorrected errors. Also, an Interference Index was calculated (Colors/Dot time scores). English language normative data used to calculate standard performances were derived using Canadian data (Troyer et al., Reference Troyer, Leach and Strauss2006).

Procedure

The study was approved by the Monash University Human Research Ethics Committee and was conducted in accordance with the Declaration of Helsinki. All participants volunteered and did not receive financial compensation. Participants were assessed individually in a quiet, distraction-free room in their homes. Home visits were offered as a means of maximizing recruitment potential, participant comfort, and reducing the cost and burden of travel. The first author contacted nine Greek social clubs throughout the Melbourne metropolitan area and was subsequently invited to present information to attendees relating to the normative studies (Staios et al., Reference Staios, Kosmidis, Kokkinis, Papadopoulos, Nielsen, Kalinowski, March and Stolwyk2023a, b). Following a general introduction, individuals who showed interest in participation were provided with a plain language statement and allowed to ask additional questions. Those interested in participating provided contact details and a time was arranged to complete the assessment battery. AD participants and their next of kin were informed about the present study by their treating physician(s) during routine clinic attendance and were provided with a plain language statement. Individuals interested in taking part in the study consented to being contacted by the research team to complete the assessment battery. All participants provided informed consent. In instances where individuals with AD were unable to provide informed consent, their next of kin provided consent of their behalf.

Prior to completing cognitive assessment measures, all participants were initially screened using the Greek language adaptions of the MMSE, GDS-15, and GAI, and underwent a semi-structured clinical interview and provided information regarding demographics (e.g., date of birth, education level, years in Australia), medical history (e.g., history of head injury with loss of consciousness, psychiatric or neurological diagnoses), and cultural factors (e.g., age of immigration, English/Greek and/or other language skills). Healthy participants also provided information regarding self-reported changes in activities of daily living and memory.

The data presented were collected between May 2018 to February 2020. All tests were administered in Greek by two registered bilingual Greek-English-speaking clinical neuropsychologists. Testing occurred over two sessions, lasting approximately 2 hr in total for healthy participants. Healthy participants completed the test battery reported in the materials section. AD participants completed a shortened test battery, consisting of the following tests: WAIS-IV GR Selected Subtests: Block Design, Matrix Reasoning, Similarities, Digit Span; HVLT-R; WMS-IV Selected Subtests: Visual Reproduction; Greek Story Memory Test; Greek Naming Test; Semantic Verbal Fluency: Animals and Supermarket, and the CTT. Testing occurred over one session and lasted approximately 1 hr for AD participants.

Data analysis

Statistical analyses were performed using the Statistical Package for the Social Sciences (SPSS), Version 22.0. Independent sample t tests were used to examine group differences between healthy and AD participants on age, years of education, time in Australia, MMSE, CDR, GDS-15, and GAI scores. The chi-square test of independence was used to examine group differences regarding the distribution of sex. Cohen’s d was used to calculate effect sizes with d values = 0–0.2 small, 0.2–0.5 medium, and >0.8 large (Cohen, Reference Cohen1988).

Comparison of standardized performances and impairment rates using Greek Australian versus English language normative data

Comparison of standardized performances and impairment rates yielded from newly developed Greek Australian normative data and English language normative data was examined only within healthy participants (N = 90). All test raw scores were converted to age adjusted (and where available education adjusted) scaled scores. A series of paired samples t tests were used to compare differences between normative data sets on neuropsychological measures within healthy participants. An alpha level of less than .05 two-tailed was considered statistically significant. Cohen’s d was again used to calculate effects sizes.

The number of participants classified as impaired versus intact were calculated separately using Greek Australian normative data and English language normative data. For both normative data sets, a performance of 1.5 standard deviations below the mean was used to indicate impairment. The relative proportion of the Greek Australian sample scoring 1.5 standard deviations below the mean was compared to the theoretically expected proportion of 7% of the sample that should fall within the impaired range. A comparison of the proportion of impairment rates when using Greek Australian versus English language normative data were examined using McNemar’s tests (McNemar, Reference McNemar1947).

Sensitivity and specificity of AD-related cognitive impairment detection within older Greek Australians

A series of independent sample t tests were first conducted to compare raw score performances between healthy and AD participants on all neuropsychological measures. An alpha level of less than .05 two-tailed was considered statistically significant. Cohen’s d was again used to calculate effect sizes for all neuropsychological measures. A receiver operating characteristic (ROC) curve analysis was used to identify optimal cutoff scores and examine the sensitivity and specificity of the neuropsychological tests using AD participants as the reference standard. Sensitivity and specificity were estimated by the area under the curve (AUC), with a 95% confidence interval. Sensitivity and specificity were calculated for the overall sample with AUC values = < 0.5–0.6 poor, = 0.7–0.8 acceptable, = 0.8–0.9 excellent, and = > 0.9 outstanding (Hosmer et al., Reference Hosmer, Lemeshow and Sturdivant2013). Cut scores for each measure were calculated in order to correspond to the optimal balance between sensitivity and specificity. Sensitivity and specificity of 5th and 10th percentile cut scores according to age adjusted normative data for each measure, were also calculated.

Results

Participants

One hundred and one healthy participants were initially recruited. Five participants were excluded from the study; three scored below cutoff for cognitive impairment on the Greek version of the MMSE; two scored above the cutoff for depression on the GDS-15. A further six participants withdrew due to time constraints as a result of caregiving duties or upcoming medical procedures. The final healthy sample consisted of 90 community-dwelling individuals. A total of 24 participants with a diagnosis of AD were invited to take part in the study. Four participants withdrew and did not complete testing due to a decline in physical and/or mental health. The final sample consisted of 20 participants with a confirmed diagnosis of AD.

Demographic data for the healthy and AD participants are presented in Table 1. In summary, AD and healthy participants did not differ with respect to age, years of education, sex, or years in Australia. In contrast, there was a significant difference between healthy and AD participants on the MMSE, with the latter group achieving significantly lower scores. All AD participants were classified as moderately impaired on the CDR, while healthy participants fell within normal limits. AD participants endorsed more symptoms of depression than healthy participants, however, no participant in either group reported clinically significant levels of depression or anxiety.

Table 1. Demographic data for healthy and AD group

AD = Alzheimer’s disease, CDR = Clinical Dementia Rating Scale, GDS = Geriatric Depression Scale, GAI = Geriatric Anxiety Inventory, MMSE = Mini-Mental State Exam, n = number, SD = standard deviation.

Comparison of standardized performances and impairment rates calculated using Greek Australian versus English language normative data

Standardized performances of our healthy sample, calculated separately using Greek Australian and English language normative data, are presented in Table 2. In summary, mean performances yielded using English language normative data were significantly lower than those obtained using Greek Australian normative data for all tests, with medium to large effect sizes noted. The magnitude of differences between normative data sets generally ranged between one to two standard deviations below the mean when using English language normative data.

Table 2. Comparison of scaled scores calculated for healthy participants using Greek Australian versus majority English language norms (N = 90)

d = effect size, IQR = interquartile range, M = mean, SD = standard deviation.

Rates of impairment across neuropsychological tests, calculated separately using Greek Australian and English language normative data, are presented in Figure 1 (WAIS-IV subtests) and Figure 2 (all other neuropsychological measures), corresponding to a 1.5 standard deviation criterion. Impairment rates derived from the Greek Australian normative data using a 1.5 standard deviation criterion showed that rates of impairment generally fell within the expected 7% range. In contrast, impairment rates for all tests derived using English language normative data (except for the HVLT-R Discrimination Index) were significantly higher and ranged from 11-66% (all ps < .05).

Figure 1. Comparison of rates of impairment on WAIS-IV core subtests calculated using Greek-Australian versus majority English language norms. SI = Similarities, VC = Vocabulary, IN = Information, BD = Block Design, MR = Matrix Reasoning, VP = Visual Puzzles, DS = Digit Span, DSF = Digit Span Forward, DSB = Digit Span Backward, DSS = Digit Span Sequencing, AR = Arithmetic, SS = Symbol Search, CD = Coding.

Figure 2. Comparison of rates of impairment across a range of neuropsychological measures calculated using Greek-Australian versus majority English language normative data. HVLT = Hopkins Verbal Learning Test; WMS = Wechsler Memory Scale; VF = Verbal Fluency; CTT = Color Trails Test; VIC = Victoria Stroop.

Sensitivity and specificity of AD-related cognitive impairment detection within older Greek-Australians

Raw neuropsychological test data for all tests are presented separately for healthy and AD participant groups in Table 3. In summary, mean scores for the AD group were significantly lower compared to the healthy group (all ps < .05), with all tests displaying medium to large effect sizes. Results from the ROC curve analyses are presented in Table 4 with AUC ranging from .721 to .999 across tests. More specifically, outstanding sensitivity and specificity (AUC > .90) were noted for Block Design and Digit Span (all conditions), Hopkins Verbal Learning Test-Revised (Total Score, Delayed Recall, Percentage Retained, Discrimination Index), Greek Story Memory Test (all conditions), Visual Reproduction II and Semantic Fluency (all conditions). Excellent sensitivity and specificity (AUC > .80) were noted for Matrix Reasoning, Hopkins Verbal Learning Test-Revised (Learning), Visual Reproduction I, and Visual Reproduction Recognition. In contrast, acceptable sensitivity and specificity (AUC > .70) were noted for the Similarities subtest and CTT (all conditions). Sensitivity ranged from .450 to 1.00, with a mean sensitivity of .956 across all measures. Specificity ranged from .511 to .978, with a mean specificity of .844 across all measures. Sensitivity and specificity of 5th percentile and 10th percentile cut scores are also presented in Table 4. Across measures, sensitivity ranged from .200 to 1.00, with a mean sensitivity of .692. Specificity ranged from .856 to 1.00, with a mean specificity of .926.

Table 3. Raw neuropsychological test scores between healthy and AD participants

d = effect size, IQR = interquartile range, M = mean, SD = standard deviation.

Table 4. Sensitivity and specificity of neuropsychological measures and cut points based on ROC curve analysis

AUC = area under the curve, ROC = receiver operating characteristic, SN = sensitivity, SP = specificity.

Discussion

This study aimed to compare standard performances and impairment rates of a healthy Greek Australian older adult sample calculated using either Greek Australian or English language normative data. We also sought to examine whether cut scores could be identified, capable of sensitively and specifically distinguishing between healthy Greek Australians from those with a diagnosis of AD. Consistent with previous research in other ethnic minority groups (Heaton et al., Reference Heaton, Taylor, Manly, Tulsky, Saklofske, Chelune, Heaton, Ivnik, Bornstein, Prifitera and Ledbetter2003), the application of English language normative data within our Greek Australian sample resulted in significantly lower performances across all tests and higher impairment, with medium to large effect sizes noted across most measures. In contrast, when Greek Australian normative data were used, performances yielded typical impairments rates and distributions reflecting expected interindividual variability within a healthy population (Schretlen et al., Reference Schretlen, Munro, Anthony and Pearlson2003), with approximately 7% of the sample falling below the 1.5 standard deviation cutoff. While statistical comparisons of education between standardization samples were not possible, the level of education observed within the English language normative samples were substantially higher when compared to the Greek Australian sample; a factor that most likely contributed to the magnitude of the current findings. Similar findings have been noted in previous research, where the application of English language normative data to culturally diverse individuals with ≤6 years of education has been noted to result in high misclassification rates (Cherner et al., Reference Cherner, Suarez, Lazzaretto, Fortuny, Mindt, Dawes, Marcotte, Grant and Heaton2007). Consistent with previous research, the present findings demonstrate that the use of representative normative data resulted in superior performance of the cognitive measures, with the ability to accurately classify this group of educationally disadvantaged older adults and reduce rates of misclassification (Díaz-Santos et al., Reference Díaz-Santos, Suárez, Marquine, Umlauf, Rivera Mindt, Fortuny, Heaton, R. and Cherner2021; Marquine et al., Reference Marquine, Morlett Paredes, Madriaga, Blumstein, Umlauf, Kamalyan and Cherner2020; Marquine et al., Reference Marquine, Yassai-Gonzalez, Perez-Tejada, Umlauf, Kamalyan, Morlett Paredes, Rivera Mindt, Artiola, Fortuny, Cherner and Heaton2021; Scott et al., Reference Scott, Morlett Paredes, Taylor, Umlauf, Fortuny, Heaton, R., Cherner, Marquine and Rivera Mindt2021; Suarez et al., Reference Suarez, Díaz-Santos, Marquine, Kamalyan, Mindt, Umlauf, Heaton, Grant and Cherner2021).

We found that the application of English language normative data resulted in significantly lower mean scores and over 57% of healthy participants were classified as impaired on all WAIS-IV verbal subtests. In light of previous research findings, it was not surprising that the use of nonrepresentative normative data resulted in high impairment rates on verbal subtests given the level of education observed within the present sample (Razani et al., Reference Razani, Murcia, Tabares and Wong2007; Shuttleworth-Edwards et al., Reference Shuttleworth-Edwards, Kemp, Rust, Muirhead, Hartman and Radloff2004). In contrast, while mean scores on WAIS-IV nonverbal subtests were marginally higher (except for Matrix Reasoning) in comparison to verbal subtests, the use of English language normative data underestimated performances and impairment rates were still unacceptably high. While past research has established that nonverbal measures are also prone to the effects of culture and education, their effects may be less pronounced as such measures are less dependent on language (Ardila et al., Reference Ardila, Ostrosky-Solis, Rosselli and Gómez2000). Furthermore, high impairment rates when using English language normative data were also noted on tasks with a timed component, supporting the premise that attitudes toward speeded tasks can be impacted by cultural orientations toward time (Agranovich et al., Reference Agranovich, Panter, Puente and Touradji2011; Messinis et al., Reference Messinis, Malegiannaki, Christodoulou, Panagiotopoulos and Papathanasopoulos2011). Interestingly, the application of English language normative data resulted in misclassifying 33% of healthy participants on WMS-IV Visual Reproduction I. This finding is also consistent with previous research examining drawing tests in older low educated immigrant groups, which have consistently shown that completing such tests is particularly difficult in these populations due to limited experience with using pencils, remembering graphic symbols, and the organization and analysis of visuospatial information (Hong et al., Reference Hong, Yoon, Shim, Cho, Lee, Kim and Yang2011; Nielsen & Jørgensen, Reference Nielsen and Jørgensen2013; Staios et al., Reference Staios, Nielsen, Kosmidis, Papadopoulos, Kokkinias, Velakoulis, Tsiaras, March and Stolwyk2022). Finally, no differences were observed on the HVLT-R Discrimination Index between normative data sets. This finding suggests that while cultural and education factors appeared to significantly impact learning and free recall, these factors may have a relatively lower impact on the retention of information and ability to discriminate between true and false positives items. However, further research is needed to confirm these outcomes.

Overall, we were able to establish high sensitivity and specificity in our sample, while also providing cut scores for a broad range of neuropsychological measures. Results showed that Greek Australian participants with AD performed poorer than their healthy counterparts on measures of verbal and visual memory, language, visuospatial skills, and executive functions, with most neuropsychological measures providing robust sensitivity and specificity. Among the neuropsychological tests used in this study, Block Design, Digit Span (total and backward), HVLT-R, Greek Story Memory Test, Greek Naming Test, and Semantic Verbal Fluency were found to have the highest sensitivity and specificity (AUC > .90), followed by Visual Reproduction (AUC > .80). Consistent with previous research (Sala et al., Reference Sala, Illán-Gala, Alcolea, Sánchez-Saudinós, Salgado, Morenas-Rodríguez, Subirana, Videla, Clarimón, Carmona-Iragui, Ribosa-Nogué, Blesa, Fortea and Lleó2017; Salimi et al., Reference Salimi, Irish, Foxe, Hodges, Piguet and Burrell2018; Weissberger et al., Reference Weissberger, Strong, Stefanidis, Summers, Bondi and Stricker2017), higher sensitivity and specificity was found on tasks assessing verbal and visual memory in discriminating AD, given the profound deficits that those diagnosed with AD exhibit in these cognitive domains (Clarens et al., Reference Clarens, Crivelli, Calandri, Chrem Méndez, Martin, Russo and Allegri2022; Ramirez-Gomez et al., Reference Ramirez-Gomez, Zheng, Reed, Kramer, Mungas, Zarow and Chui2017). Furthermore, measures of executive functioning, namely working memory and verbal fluency, showed equivalent sensitivity and specificity. Previous research examining dementia profiles in ethnic minority groups have noted that verbal memory deficits outweigh those observed in visuospatial skills, language and executive functioning, in the initial stage of disease progression (Barnes-Marrero et al., Reference Barnes-Marrero, Horter, Hayden, Patel, Mendoza and Castillo2022; Nielsen et al., Reference Nielsen, Segers, Vanderaspoilden, Bekkhus-Wetterberg, Minthon, Pissiota, Bjørkløf, Beinhoff, Tsolaki, Gkioka and Waldemar2018). In this context, the degree of executive deficits observed in our AD group is likely attributed to the fact that they were moderately impaired, where such deficits are commonly observed (Weintraub et al., Reference Weintraub, Wicklund and Salmon2012).

Moreover, of all the neuropsychological measures administered, the Similarities subtest and the CTT showed low specificity (AUC = <.80). Poor specificity values noted on the Similarities subtest were likely due to factors such as level and quality of education obtained in Greece pre-immigration and acculturation (Staios, Reference Staios and Irani2022). Regarding the CTT, results revealed that the total time score for both conditions was found to yield a high proportion of false positive results, as indicated by low specificity values, leading to a misidentification of approximately half of the healthy participants. Similar findings have previously been noted in a sample of educationally disadvantaged Brazilian older adults, where both conditions of the CTT time score for were found to lack sensitivity and specificity when compared to AD participants (Araujo et al., Reference Araujo, Nielsen, Barca, Engedal, Marinho, Deslandes, Coutinho and Laks2020). These findings are likely due to several factors, including no previous experience with psychometric testing, level and quality of education, and cultural orientation toward time and speeded tasks (Agranovich et al., Reference Agranovich, Panter, Puente and Touradji2011; Al-Jawahiri & Nielsen, Reference Al-Jawahiri and Nielsen2021; Messinis et al., Reference Messinis, Malegiannaki, Christodoulou, Panagiotopoulos and Papathanasopoulos2011). In light of these findings, in instances where clinicians are required to carry out evaluations of Greek Australian elders with moderate dementia, we recommend using the following measures: WAIS-IV Block Design and Digit Span, Hopkins Verbal Learning Test-Revised, Greek Story Memory Test, Greek Naming Test, and Verbal Fluency. The aforementioned measures displayed excellent sensitivity and specificity in discriminating health from AD participants and are also appropriate for testing several cognitive domains known to be impacted by AD (Lezak et al., Reference Lezak, Howieson, Bigler and Tranel2012).

Finally, we aimed to identify cut scores for a broad range of neuropsychological measures, based on the optimal balance between sensitivity and specificity, and compare this to traditionally used benchmarks of 5th and 10th percentiles. Overall, optimal cut scores yielded excellent results for the assessment of moderately impaired individuals with AD and are now available for research and clinical use. Inspection of the data concerning the 5th percentile revealed that this cutoff compromised sensitivity across most tasks. In contrast, the cut scores across both the suggested optimal and 10th percentile ranges remained approximately the same. However, a notable difference was observed for the Similarities and Matrix Reasoning subtests, Visual Reproduction (recognition), and CTT, where the use of 5th and 10th percentiles corresponded to a higher discrepancy between sensitivity and specificity in favor of the latter in comparison with the optimal cutoff scores. This finding may provide useful information in clinical practice, as it allows the selection of the cutoff score with the most suitable combination between sensitivity and specificity depending on the clinical question at hand. In other words, a high sensitivity score may be preferable when older adults at higher risk for dementia need to be identified, while a high specificity score could be more useful when confirmation of AD is necessary. In summary, the selection of the cutoff score for these specific tests could provide different outcomes according to the goal of the clinical assessment.

This study had several limitations. First, participants were selected through convenience sampling and consisted of primary school educated individuals. Therefore, results are not generalizable or appropriate for clinical use when assessing Greek Australians with higher levels of education. Second, the fact that our clinical group comprised moderately impaired AD participants, as can be inferred by the discrepancy in MMSE and CDR scores between the two groups, may partially explain the excellent sensitivity and specificity observed in several neuropsychological tests. Therefore, the cut scores derived in this study should only be applied to Greek Australians presenting with moderate to severe forms of dementia. In cases where milder forms of dementia are suspected, we advise referring to our normative studies and using these data to calculate probable levels of impairment (Staios et al., Reference Staios, Kosmidis, Kokkinis, Papadopoulos, Nielsen, Kalinowski, March and Stolwyk2023a, b). Despite our best efforts to recruit participants with mild cognitive impairment, we were unable to do so. Greek Australians and other culturally diverse groups continue to be underrepresented in clinical research, relative to their Anglo Australian counterparts (Low et al., Reference Low, Barcenilla-Wong and Brijnath2019). Factors such as stigma, limited access to culturally appropriate resources and treatment services continue to be an issue facing culturally diverse Australians, leading to underutilization of health care services (LoGiudice et al., Reference LoGiudice, Hassett, Cook, Flicker and Ames2001; Low et al., Reference Low, Anstey, Lackersteen, Camit, Harrison, Draper and Brodaty2010; Phillipson et al., Reference Phillipson, Magee, Jones, Reis and Skladzien2015). As a result, a combination of these factors impacted our ability to recruit participants with early-stage AD. Future research would benefit from exploring the sensitivity and specificity of the tests used in the present study within a broader and milder range of clinical presentations, as well as their ability to differentiate between a range of neurocognitive disorders within the Greek Australian community.

In conclusion, we believe our findings represent an important contribution to the field of clinical neuropsychology. We have demonstrated how using English language normative data within healthy educationally disadvantaged ethnic minority groups can result in erroneous diagnostic outcomes. We have also demonstrated how use of demographically focused normative data and alternate cut points can help ameliorate this issue. We anticipate that the methodology employed in the present study may be used as a template for other ethnic minority groups to improve cross-cultural neuropsychological practice and test development internationally. Finally, while culturally specific tests and normative data may facilitate accuracy of testing outcomes, they should not be viewed as the only solution to improving the validity of cognitive assessment within underrepresented groups (Ardila, Reference Ardila2005). The need for increasing cultural competence, the development of culturally considered clinical guidelines, ongoing professional development, and wider representation within the field of neuropsychology are necessary (Rivera Mindt et al., Reference Rivera Mindt, Byrd, Saez and Manly2010).

Funding statement

This research was supported by Fronditha Care, the Australasian Hellenic Educational Progressive Association, Marathon Foods, Anna Timou, the Australian Hellenic Golf Federation, the Hellenic Women’s Cultural Association, Kirsty Chiaplias, and the Thessaloniki Association.

Competing interests

None.

References

Agranovich, A. V., Panter, A. T., Puente, A. E., & Touradji, P. (2011). The culture of time in neuropsychological assessment exploring the effects of culture-specific time attitudes on timed test performance in Russian and American samples. Journal of the International Neuropsychological Society, 17 (4), 692701. https://doi.org/10.1017/S1355617711000592 CrossRefGoogle ScholarPubMed
Al-Jawahiri, F., & Nielsen, T. R. (2021). Effects of acculturation on the cross-cultural neuropsychological test battery (CNTB) in a culturally and linguistically diverse population in Denmark. Archives of Clinical Neuropsychology, 36 (3), 381393. https://doi.org/10.1093/arclin/acz083 CrossRefGoogle Scholar
American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders. (5th edn). American Psychiatric Association.Google Scholar
Araujo, N. B., Nielsen, T. R., Barca, M. L., Engedal, K., Marinho, V., Deslandes, A. C., Coutinho, E. S., & Laks, J. (2020). Brazilian version of the European Cross-Cultural Neuropsychological Test Battery (CNTB-BR): diagnostic accuracy across schooling levels. Revista Brasileira de Psiquiatria, 42 (3), 286294. https://doi.org/10.1590/1516-4446-2019-0539 CrossRefGoogle ScholarPubMed
Ardila, A. (2005). Cultural values underlying psychometric cognitive testing. Neuropsychology Review, 15 (4), 185195. https://doi.org/10.1007/s11065-005-9180-y CrossRefGoogle ScholarPubMed
Ardila, A., Ostrosky-Solis, F., Rosselli, M., & Gómez, C. (2000). Age-related cognitive decline during normal aging: The complex effect of education. Archives of Clinical Neuropsychology, 15 (6), 495513.Google ScholarPubMed
Australian Bureau of Statistic (2016). People in Australia who were born in Greece. Canberra: Author.Google Scholar
Australian Institute of Health and Welfare (2018). Dementia in Australia. Canberra: Author.Google Scholar
Australian Institute of Health and Welfare (2021). Older Australians. Canberra: Author.Google Scholar
Barnes-Marrero, I., Horter, L., Hayden, J. D., Patel, N. C., Mendoza, L., & Castillo, L. (2022). Diagnostic accuracy of the repeatable battery of the assessment of neuropsychological status update, Spanish version, in predicting Alzheimer’s disease among Hispanic older adults in the United States reporting memory problems. Applied Neuropsychology: Adult, 29 (4), 509519. https://doi.org/10.1080/23279095.2020.1777554 CrossRefGoogle ScholarPubMed
Blakemore, A., Kenning, C., Mirza, N., Daker-White, G., Panagioti, M., & Waheed, W. (2018). Dementia in UK South Asians: A scoping review of the literature. BMJ Open, 8 (4), e020290. https://doi.org/10.1136/bmjopen-2017-020290 CrossRefGoogle ScholarPubMed
Brandt, J., & Benedict, R. H. B. (2001). Hopkins verbal learning test – revised. Administration manual. Psychological Assessment Resources.Google Scholar
Cherner, M., Suarez, P., Lazzaretto, D., Fortuny, L. A., Mindt, M. R., Dawes, S., Marcotte, T., Grant, I., Heaton, R., & HNRC group (2007). Demographically corrected norms for the brief visuospatial memory test-revised and hopkins verbal learning test-revised in monolingual Spanish speakers from the U.S.-Mexico border region. Archives of Clinical Neuropsychology, 22 (3), 343353. https://doi.org/10.1016/j.acn.2007.01.009 CrossRefGoogle ScholarPubMed
Clarens, M. F., Crivelli, L., Calandri, I., Chrem Méndez, P., Martin, M. E., Russo, M. J., & Allegri, R. F. (2022). Neuropsychological profile of Alzheimer’s disease based on amyloid biomarker findings results from a South American cohort. Applied Neuropsychology: Adult, 29(3), 345350. https://doi.org/10.1080/23279095.2020.1756816 CrossRefGoogle ScholarPubMed
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd edn). Hillside, NJ: Lawrence Erlbaum Associates.Google Scholar
D’Elia, L. F., Satz, P., Uchiyama, C. L., & White, T. (1996). Color Trails Test. Odessa, Florida: Psychological Assessment Resources.Google Scholar
Daugherty, J. C., Puente, A. E., Fasfous, A. F., Hidalgo-Ruzzante, N., & Pérez-Garcia, M. (2017). Diagnostic mistakes of culturally diverse individuals when using North American neuropsychological tests. Applied Neuropsychology: Adult, 24 (1), 1622. https://doi.org/10.1080/23279095.2015.1036992 CrossRefGoogle ScholarPubMed
Díaz-Santos, M., Suárez, P. A., Marquine, M. J., Umlauf, A., Rivera Mindt, M., Fortuny, Artiola I., Heaton, L., R., K., & Cherner, M. (2021). Updated demographically adjusted norms for the brief visuospatial memory test-revised and hopkins verbal learning test-revised in Spanish-speakers from the U.S.-Mexico border region: The NP-NUMBRS project. The Clinical Neuropsychologist, 35 (2), 374395. https://doi.org/10.1080/13854046.2020.1861329 CrossRefGoogle ScholarPubMed
Dubois, B., Feldman, H. H., Jacova, C., DeKosky, S. T., Barberger-Gateau, P., Cummings, J., et al. (2007). Research criteria for the diagnosis of Alzheimer’s disease: Revising the NINCDS-ADRDA criteria. The Lancet Neurology, 6 (8), 734746. https://doi.org/10.1016/s1474-4422(07)70178-3 CrossRefGoogle ScholarPubMed
Fanany, R., & Avgoulas, M. I. (2019). Greek Identity in Australia. In Ratuva, S. (Eds.), The Palgrave Handbook of Ethnicity (pp. 11861199). Palgrave Macmillan.Google Scholar
Fountoulakis, K. N., Tsolaki, M., Chantzi, H., & Kazis, A. (2000). Mini mental state examination (MMSE): A validation study in Greece. American Journal of Alzheimer’s Disease, 15 (6), 342345. https://doi.org/10.1177/153331750001500604 CrossRefGoogle Scholar
Fountoulakis, K. N., Tsolaki, M., Iacovides, A., Yesavage, J., O’Hara, R., Kazis, A., et al. (1999). The validation of the short form of the geriatric depression scale (GDS) in Greece. Aging: Clinical and Experimental Research, 11 (6), 367372. https://doi.org/10.1007/bf03339814 Google ScholarPubMed
Franzen, S., Papma, J. M., van den Berg, E., & Nielsen, T. R. (2021). Cross-cultural neuropsychological assessment in the European union: A Delphi expert study. Archives of Clinical Neuropsychology, 36 (5), 815830. https://doi.org/10.1093/arclin/acaa083 Google ScholarPubMed
Fratti, S., Bowden, S.C., & Pino, O. (2011). Diagnostic memory assessment in Italian-Born Australians. International Psychogeriatrics, 23:7,1133–43.CrossRefGoogle Scholar
Heaton, R. K., Taylor, M. J., & Manly, J. (2003). Demographic effects and use of demographically corrected norms with the WAIS-III and WMS-III. In Tulsky, D. S., Saklofske, D. H., Chelune, G. J., Heaton, R. K., Ivnik, R. J., Bornstein, R., Prifitera, A., & Ledbetter, M. F. (Eds.), Clinical interpretation of the WAIS-III and WMS-III (pp. 181210). Academic Press.CrossRefGoogle Scholar
Hester, R. L., Kinsella, G. J., Ong, B., & Turner, M. (2004). Hopkins verbal learning test: Normative data for older Australian adults. Australian Psychologist, 39 (3), 251255. https://doi.org/10.1080/000500604123312950 CrossRefGoogle Scholar
Hong, Y. J., Yoon, B., Shim, Y. S., Cho, A.-H., Lee, E.-S., Kim, Y.-I., & Yang, D. W. (2011). Effect of literacy and education on the visuoconstructional ability of non-demented elderly individuals. Journal of the International Neuropsychological Society, 17 (05), 934939. https://doi.org/10.1017/s1355617711000889 CrossRefGoogle ScholarPubMed
Hosmer, W. D., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (3rd ed). New Jersey: Wiley, https://doi.org/10.1002/9781118548387 CrossRefGoogle Scholar
Hughes, C. P., Berg, L., Danziger, W. L., Coben, L. A., & Martin, R. L. (1982). A new clinical scale for the staging of dementia. The British Journal of Psychiatry, 140 (6), 566572. https://doi.org/10.1192/bjp.140.6.566 CrossRefGoogle ScholarPubMed
Jupp, J. (2001). The Australian people: An encyclopedia of the nation, and their origins. New York: Cambridge University Press.Google Scholar
Kochhann, R., Varela, J. S., de Macedo Lisboa, C. S., & Chaves, M. L. F. (2010). The mini mental state examination: Review of cutoff points adjusted to school- ing in a large southern Brazilian sample. Dementia & Neuropsychologia, 4 (1), 3541. https://doi.org/10.1590/S1980-57642010DN40100006CrossRefGoogle Scholar
Kosmidis, M. H. (2017). Challenges in the neuropsychological assessment of illiterate older adults. Language, Cognition and Neuroscience, 33 (3), 373386. https://doi.org/10.1080/23273798.2017.1379 CrossRefGoogle Scholar
Kosmidis, M. H., Bozikas, V. P., & Vlahou, C. H. (2012). Neuropsychological Test Battery. Greece: Lab of Cognitive Neuroscience, School of Psychology, Aristotle University of Thessaloniki.Google Scholar
Kosmidis, M. H., Vlahou, C. H., Panagiotaki, P., & Kiosseoglou, G. (2004). The verbal fluency task in the Greek population: Normative data, and clustering and switching strategies. Journal of the International Neuropsychological Society, 10 (2), 164172. https://doi.org/10.1017/S1355617704102014 CrossRefGoogle ScholarPubMed
Lezak, M. D., Howieson, D. B., Bigler, E. D., & Tranel, D. (2012). Neuropsychological Assessment (5th edn). New York: Oxford University Press.Google Scholar
LoGiudice, D., Hassett, A., Cook, R., Flicker, L., & Ames, D. (2001). Equity of access to a memory clinic in Melbourne? Non-English speaking background attenders are more severely demented and have increased rates of psychiatric disorders. International Journal of Geriatric Psychiatry, 16 (3), 327334. https://doi.org/10.1002/gps.346 CrossRefGoogle ScholarPubMed
Low, L. F., Anstey, K. J., Lackersteen, S. M., Camit, M., Harrison, F., Draper, B., & Brodaty, H. (2010). Recognition, attitudes and Causal beliefs regarding dementia in italian, Greek and Chinese Australians. Dementia and Geriatric Cognitive Disorders, 30 (6), 499508. https://doi.org/10.1159/000321667 CrossRefGoogle ScholarPubMed
Low, L. F., Barcenilla-Wong, A. L., & Brijnath, B. (2019). Including ethnic and cultural diversity in dementia research. Medical Journal of Australia, 211 (8), 345. https://doi.org/10.5694/mja2.50353 CrossRefGoogle ScholarPubMed
Marquine, M. J., Morlett Paredes, A., Madriaga, C., Blumstein, Y., Umlauf, A., Kamalyan, L., Cherner, M., et al. (2020). Demographically-adjusted norms for selected tests of verbal fluency: Results from the neuropsychological norms for the US-Mexico border region in Spanish (NP-NUMBRS) project. The Clinical Neuropsychologist, 124. https://doi.org/10.1080/13854046.2020.1762 Google ScholarPubMed
Marquine, M. J., Yassai-Gonzalez, D., Perez-Tejada, A., Umlauf, A., Kamalyan, L., Morlett Paredes, A., Rivera Mindt, M., Artiola, I., Fortuny, L., Cherner, M., & Heaton, R. K. (2021). Demographically adjusted normative data for the Wisconsin card sorting test-64 item: Results from the neuropsychological norms for the U.S.-Mexico border region in Spanish (NP-NUMBRS) project. The Clinical Neuropsychologist, 35 (2), 339355. https://doi.org/10.1080/13854046.2019.1703042 CrossRefGoogle ScholarPubMed
McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12 (2), 153157. https://doi.org/10.1007/BF02295996 CrossRefGoogle ScholarPubMed
Messinis, L., Malegiannaki, A.-C., Christodoulou, T., Panagiotopoulos, V., & Papathanasopoulos, P. (2011). Color trails test: Normative data and criterion validity for the Greek adult population. Archives of Clinical Neuropsychology, 26 (4), 322330. https://doi.org/10.1093/arclin/acr027 CrossRefGoogle ScholarPubMed
Nielsen, T. R., Segers, K., Vanderaspoilden, V., Bekkhus-Wetterberg, P., Minthon, L., Pissiota, A., Bjørkløf, G. H., Beinhoff, U., Tsolaki, M., Gkioka, M., & Waldemar, G. (2018). Performance of middle-aged and elderly European minority and majority populations on a cross-cultural neuropsychological test battery (CNTB). The Clinical Neuropsychologist, 32 (8), 14111430. https://doi.org/10.1080/13854046.2018.1430256 CrossRefGoogle ScholarPubMed
Nielsen, T. R., & Jørgensen, K. (2013). Visuoconstructional abilities in cognitively healthy illiterate Turkish immigrants: A quantitative and qualitative investigation. The Clinical Neuropsychologist, 27 (4), 681692. https://doi.org/10.1080/13854046.2013.767379 CrossRefGoogle ScholarPubMed
Nielsen, T. R., & Waldemar, G. (2021). Cross-cultural cognitive examination in aging migrants. In M. E. Alaoui-Faris, A. Federico, & W. Grisold (Eds.), Neurology in Migrants and Refugees (1st ed., pp. 267281). Springer Nature: Switzerland.Google Scholar
Noutsos, H. (2003). The road of the camel and school: The educational policy in Greece from 1944-46. Athens: Bibliorama.Google Scholar
Pachana, N. A., Byrne, G. J., Siddle, H., Koloski, N., Harley, E., & Arnold, E. (2007). Development and validation of the geriatric anxiety inventory. International Psychogeriatrics, 19 (01), 103114.CrossRefGoogle ScholarPubMed
Pedraza, O., Clark, J. H., O’Bryant, S. E., Smith, G. E., Ivnik, R. J., Graff-Radford, N. R., Lucas, J. A., Willis, F. B., Petersen, R. C., & Lucas, J. A. (2011). Diagnostic validity of age and educa- tion corrections for the mini-mental state examination in older African Americans. Journal of the American Geriatrics Society, 60 (2), 328331. https://doi.org/10.1111/j.1532-5415.2011.03766.x CrossRefGoogle Scholar
Phillipson, L., Magee, C., Jones, S. C., Reis, S., & Skladzien, E. (2015). Dementia attitudes and help-seeking intentions: An investigation of responses to two scenarios of an experience of the early signs of dementia. Aging and Mental Health, 19 (11), 968977. https://doi.org/10.1080/13607863.2014.995588 CrossRefGoogle ScholarPubMed
Pienaar, I., Shuttleworth-Edwards, A., Klopper, C., & Radloff, S. (2016). Wechsler adult intelligence scale-fourth edition preliminary normative guidelines for educationally disadvantaged xhosa-speaking individuals. South African Journal of Psychology, 47 (2), 159170. https://doi.org/10.1177/0081246316654805 CrossRefGoogle Scholar
Plitas, A., Tucker, A., Kritikos, A., Walters, I., & Bardenhagen, F. (2009). Comparative study of the cognitive performance of Greek Australian and Greek national elderly: Implications for neuropsychological practice. Australian Psychologist, 44 (1), 2739. https://doi.org/10.1080/00050060802587694 CrossRefGoogle Scholar
Ramirez-Gomez, L., Zheng, L., Reed, B., Kramer, J., Mungas, D., Zarow, C., & Chui, H. (2017). Neuropsychological profiles differentiate Alzheimer disease from subcortical ischemic vascular dementia in an autopsy-defined cohort. Dementia and geriatric cognitive. Dementia and Geriatric Cognitive Disorders, 44 (1-2), 111. https://doi.org/10.1159/000477344 CrossRefGoogle Scholar
Razani, J., Murcia, G., Tabares, J., & Wong, J. (2007). The effects of culture on WASI test performance in ethnically diverse individuals. The Clinical Neuropsychologist, 21 (5), 776788. https://doi.org/10.1080/13854040701437481 CrossRefGoogle ScholarPubMed
Regard, M. (1981). Cognitive rigidity and flexibility: A neuropsychological study. [Unpublished doctoral dissertation]. British Columbia, Canada: University of Victoria.Google Scholar
Reitan, R. M. (1955). The relation of the trail making test to organic brain damage. Journal of Consulting Psychology, 19 (5), 393394. https://doi.org/10.1037/h0044509 CrossRefGoogle ScholarPubMed
Rivera Mindt, M., Byrd, D., Saez, P., & Manly, J. (2010). Increasing culturally competent neuropsychological services for ethnic minority populations: A call to action. The Clinical Neuropsychologist, 24 (3), 429453. https://doi.org/10.1080/13854040903058960 CrossRefGoogle ScholarPubMed
Rivera Mindt, M., Marquine, M. J., Aghvinian, M., Paredes, A. M., Kamalyan, L., Suárez, P., Heaton, A., Scott, T. M., Gooding, A., Diaz-Santos, M., Umlauf, A., Taylor, M. J., Artiola I.Fortuny, L., Heaton, R. K., & Cherner, M. (2020). The Neuropsychological Norms for the U.S.-Mexico border region in Spanish (NP-NUMBRS) project: Overview and considerations for life span research and evidence-based practice. The Clinical Neuropsychologist, 115. https://doi.org/10.1080/13854046.2020.1794 Google ScholarPubMed
Sala, I., Illán-Gala, I., Alcolea, D., Sánchez-Saudinós, M. B., Salgado, S. A., Morenas-Rodríguez, E., Subirana, A., Videla, L., Clarimón, J., Carmona-Iragui, M. D., Ribosa-Nogué, R., Blesa, R., Fortea, J., & Lleó, A. (2017). Diagnostic and prognostic value of the combination of two measures of verbal memory in mild cognitive impairment due to Alzheimer’s disease. Journal of Alzheimer’s. Journal of Alzheimer’s Disease, 58 (3), 909918.CrossRefGoogle Scholar
Salimi, S., Irish, M., Foxe, D., Hodges, J. R., Piguet, O., & Burrell, J. R. (2018). Can visuospatial measures improve the diagnosis of Alzheimer’s disease? Alzheimer’s & dementia: Diagnosis. Assessment & Disease Monitoring, 10 (1), 6674. https://doi.org/10.1016/j.dadm.2017.10.004 Google Scholar
Schretlen, D. J., Munro, C. A., Anthony, J. C., & Pearlson, G. D. (2003). Examining the range of normal intraindividual variability in neuropsychological test performance. Journal of the International Neuropsychological Society, 9 (06), 864870. https://doi.org/10.1017/s1355617703960061 CrossRefGoogle ScholarPubMed
Scott, T. M., Morlett Paredes, A., Taylor, M. J., Umlauf, A., Fortuny, Artiola I., Heaton, L., R., K., Cherner, M., Marquine, M. J., & Rivera Mindt, M. (2021). Demographically-adjusted norms for the WAIS-R block design and arithmetic subtests: Results from the neuropsychological norms for the US-Mexico border region in spanish (NP-NUMBRS) project. The Clinical neuropsychologist, 35 (2), 419432. https://doi.org/10.1080/13854046.2019.1707285 CrossRefGoogle ScholarPubMed
Shuttleworth-Edwards, A. B. (2016). Generally representative is representative of none: Commentary on the pitfalls of IQ test standardization in multicultural settings. The Clinical Neuropsychologist, 30 (7), 975998.CrossRefGoogle ScholarPubMed
Shuttleworth-Edwards, A. B., Kemp, R. D., Rust, A. L., Muirhead, J. G. L., Hartman, N. P., & Radloff, S. E. (2004). Cross-cultural effects on IQ test performance: A review and preliminary normative indications on WAIS-III test performance. Journal of Clinical and Experimental Neuropsychology, 26 (7), 903920. https://doi.org/10.1080/13803390490510824 CrossRefGoogle ScholarPubMed
Staios, M. (2022). Culturally aware neuropsychological assessment with Greek immigrants. In Irani, F. (Eds.), Cultural diversity in neuropsychological assessment (pp. 341353). Routledge Inc.CrossRefGoogle Scholar
Staios, M., Kosmidis, M. H., Kokkinis, N., Papadopoulos, A., Nielsen, T. R., Kalinowski, P., March, E., & Stolwyk, R. J. (2023a). The Greek Australian neuropsychological normative study: Neuropsychological tests & norms for Greek Australians aged 70–85 years. Australian Psychologist, 115. https://doi.org/10.1080/00050067.2022.2151337 Google Scholar
Staios, M., Kosmidis, M. H., Nielsen, T. R., Papadopoulos, A., Kokkinis, N., Stogiannidou, A., March, E., & Stolwyk, R. J. (2023b). The wechsler adult intelligence scale-fourth edition, Greek adaptation (WAIS-IV GR): Confirmatory factor analysis and specific reference group normative data for Greek Australian older adults. Australian Psychologist, 116. https://doi.org/10.1080/00050067.2023.2179387 Google Scholar
Staios, M., Nielsen, T. R., Kosmidis, M. H., Papadopoulos, A., Kokkinias, A., Velakoulis, D., Tsiaras, Y., March, E., & Stolwyk, R. J. (2022). Validity of Visuoconstructional Assessment Methods within Healthy Elderly Greek Australians: Quantitative and Error Analysis. Archives of Clinical Neuropsychology, 38 (4), 598607. https://doi.org/10.1093/arclin/acac091 CrossRefGoogle Scholar
Suarez, P. A., Díaz-Santos, M., Marquine, M. J., Kamalyan, L., Mindt, M. R., Umlauf, A., Heaton, R. K., Grant, I., & Cherner, M. (2021). Demographically adjusted norms for the trail making test in native Spanish speakers: Results from the neuropsychological norms for the US-Mexico border region in Spanish (NP-NUMBRS) project. The Clinical neuropsychologist, 35 (2), 308323. https://doi.org/10.1080/13854046.2020.1800099 CrossRefGoogle ScholarPubMed
Troyer, A. K., Leach, L., & Strauss, E. (2006). Aging and response inhibition: Normative data for the victoria stroop test. Aging, Neuropsychology, and Cognition, 13 (1), 2035. https://doi.org/10.1080/138255890968187 CrossRefGoogle ScholarPubMed
Wallace, K., Berry, J., & Shores, E. A. (2018). The Chinese Australian Neuropsychological Normative Study: Neuropsychological test norms for Chinese Australians aged 55–90 years. Australian Psychologist, 53 (5), 427443. https://doi.org/10.1111/ap.12340 CrossRefGoogle Scholar
Walterfang, M., Velakoulis, D., Gibbs, A., & Lloyd, J. (2003). The NUCOG: Construction and piloting of a cognitive screening instrument in a neuropsychiatric unit. Australasian Psychiatry, 11 (3), 325329. https://doi.org/10.1046/j.1440-1665.2003.t01-1-00558.x CrossRefGoogle Scholar
Wardill, T., Anderson, V., Graham, A., & Perre, C. (2009). Normative data for older Australians. Australia: Box Hill, Monash University.Google Scholar
Wechsler, D. (2008). Wechsler adult intelligence scale-fourth edition. San Antonio, Texas: Pearson.Google Scholar
Wechsler, D. (2009). Wechsler memory scale-fourth edition. San Antonio, Texas: Pearson.Google Scholar
Wechsler, D. (2014). Wechsler adult intelligence scale-fourth edition, greek adaptation. Athens: Motibo Publishing.Google Scholar
Weintraub, S., Wicklund, A. H., & Salmon, D. P. (2012). The neuropsychological profile of alzheimer disease. Cold Spring Harbor Perspectives in Medicine, 2 (4), a006171. https://doi.org/10.1101/cshperspect.a00617 CrossRefGoogle ScholarPubMed
Weissberger, G. H., Strong, J. V., Stefanidis, K. B., Summers, M. J., Bondi, M. W., & Stricker, N. H. (2017). Diagnostic accuracy of memory measures in Alzheimer’s dementia and mild cognitive impairment: A systematic review and. Neuropsychology Review, 27 (4), 354388. https://doi.org/10.1007/s11065-017-9360-6 CrossRefGoogle ScholarPubMed
World Health Organisation (2022). Dementia, Geneva: Author.Google Scholar
Figure 0

Table 1. Demographic data for healthy and AD group

Figure 1

Table 2. Comparison of scaled scores calculated for healthy participants using Greek Australian versus majority English language norms (N = 90)

Figure 2

Figure 1. Comparison of rates of impairment on WAIS-IV core subtests calculated using Greek-Australian versus majority English language norms. SI = Similarities, VC = Vocabulary, IN = Information, BD = Block Design, MR = Matrix Reasoning, VP = Visual Puzzles, DS = Digit Span, DSF = Digit Span Forward, DSB = Digit Span Backward, DSS = Digit Span Sequencing, AR = Arithmetic, SS = Symbol Search, CD = Coding.

Figure 3

Figure 2. Comparison of rates of impairment across a range of neuropsychological measures calculated using Greek-Australian versus majority English language normative data. HVLT = Hopkins Verbal Learning Test; WMS = Wechsler Memory Scale; VF = Verbal Fluency; CTT = Color Trails Test; VIC = Victoria Stroop.

Figure 4

Table 3. Raw neuropsychological test scores between healthy and AD participants

Figure 5

Table 4. Sensitivity and specificity of neuropsychological measures and cut points based on ROC curve analysis