Performance validity tests (PVTs) provide a methodological approach to detecting credible neurocognitive performances. This proves invaluable to the diagnostic process, as it allows neuropsychologists to objectively determine if an evaluation reflects a patient’s true neurocognitive abilities or if external factors are impacting the results. However, their addition to a testing battery can increase an already lengthy evaluation. As such, there is a need for sensitive but less time intensive PVTs. The purpose of this study is to validate the Coin-in-Hand (CIH) procedure as a quick and effective PVT within a veteran population.

68 English-speaking patients were identified from an outpatient neuropsychological assessment dataset. Performances were correlated to the well- validated Reliable Digit Span (RDS), and several other soft indicators of task engagement including expanded COWAT, BVMT-False Alarms (FA), WCST Failure to Maintain Set (FTM), TOMM, and the RBANS Effort Index (EI). All participants attempted CIH and RDS, testing was discontinued if 2 or more PVTs were invalid. An AUC analysis was conducted to determine how well the CIH discriminated between valid and invalid performance and determine the tests optimal cut-off score (sensitivity > 0.90 while maintaining the highest possible specificity). Logistic Regression was conducted to determine how well the CIH predicted performance validity.

Subject mean(SD) age and education were 55.25 (16.06) and 13.41 (2.55) years, respectively. 17% female, 60% Caucasian, and 32% Black. Descriptive statistics for each of the other performance validity tests were gathered. The CIH demonstrated low diagnostic accuracy (AUC = .66; p >.05; CI = .51 -.81); a cut score of <8 resulted in a sensitivity of .96 and a specificty of .64. Logistic Regression showed that CIH performance significantly predicted performance validity (X2 = -0.93; df = 1; N = 68; p < .05), accounting for 18-28% of the variance in performance classification (Cox & Snell R2 = .18; Nagelkerke R2 = .28). It correctly classified 96% of valid performers, but only correctly classified 35% of invalid performers, with an overall correct prediction rate of 83%. A predicted chase in log odds (B= -.93) and odd ratio [Exp (B) =.40] indicated that every unit increase in CIH score was associated with a decrease probability of performance invalidity. Logistic regression was also used to calculate the probability of performance invalidity at each possible CIH score (Table 1).

Results suggests that poor performance on CIH does not necessarily equate to invalid performances, but instead, should act as a screener to cue neuropsychologists working with Veterans that additional PVTs should be considered. Overall, it was determined that CIH was able to correctly predict 35% of invalid performers and 96% of valid performers, with an overall correct prediction rate of 83%, suggesting the procedure may be too simple to be an effective standalone PVT for clinical use. These results also highlight that every correct response on the CIH was associated with a decreased probability of performance invalidity. Additionally, an AUC analysis determined the tests optimal cut off score to be <8, suggesting that shortening the procedure may be as effective as giving the full 10 trials.

Cognition has been identified as an area of priority in examining health impacts of COVID-19 infection, and evidence suggests the virus invades the brain, with potential for long-term cognitive impact. Studies utilizing screening measures have reported cognitive sequelae (e.g., attention disorder, executive dysfunction) of the post-COVID-19 condition (i.e., long-haulers). More extensive examination of cognitive difficulties via comprehensive neuropsychological assessment is critical to informing treatment for those experiencing cognitive or functional difficulties post-infection. We aimed to comprehensively evaluate cognitive resiliencies and vulnerabilities of acutely recovered COVID-19 patients, across key domains (i.e., attention, processing speed, language, visuospatial abilities, memory, executive functioning), compared to healthy controls.

Adults (N=103; aged 19-85; 69.2% female) who had COVID-19 at least three months prior (n=50) and those with no history of infection (n=53) completed demographic and health questionnaires via Qualtrics, along with measures of depressive (CES-D) and anxiety (GAD-7) symptoms, the Lawton-Brody Instrumental Activities of Daily Living (IADL) Scale, and a measure of subjective cognitive difficulties (SCD-Q). Participants (n=84) completed a teleneuropsychology assessment including a short interview and battery of neuropsychological tests assessing attention (BTA, Digit Span Forward), processing speed (DKEFS Colour Naming & Word Reading, SDMT), language (FAS, Animals, NAB Naming), visuospatial abilities (JLO, RCFT Copy), verbal and visual memory (HVLT-R, NAB Shape Learning, RCFT), and executive function (DKEFS Color-Word Interference & Switching, Digit Span Backward & Sequencing, BRIEF), and including multiple measures of cognitive effort/assessment validity (RFIT, RDS), and a self-report measure of symptom validity (SIMS). T-tests were used to examine demographic and health variables between COVID-19 and control groups. MANCOVA were used to examine group differences across each cognitive domain assessed, and across cognitive effort and symptom validity tasks, while controlling for English language status.

Group comparisons indicated that the COVID-19 group was slightly older (mean age = 40 vs. 34 yrs.; f=-2.101, p=0.04). Those who had COVID-19 reported more difficulties completing IADLs (f=2.204; p=0.03), more depressive symptoms (f=-2.299; p=0.02), and more subjective cognitive difficulties (f=-3.886; p<0.01). Examination of cognitive performance indicated a main effect of prior infection on executive function, controlling for language status (Wilks’ /\=0.817, F(6,73)=2.733, p=0.02). Specifically, having COVID-19 was associated with worse DKEFS Colour-Word Switching performance (p=0.01) and slightly higher selfreported difficulties on the BRIEF MI (p=0.04). No other significant group differences were seen across cognitive domains. There was also a main effect of COVID-19 infection on effort and symptom validity task performance (Wilks’ /\=0.705, F(10,70)=2.923, p<0.01). Specifically, prior infection was associated with higher SIMS Neurologic Impairment (p<0.01) and Amnestic Disorders (p<0.01) subscale scores, and paradoxically, slightly higher RFIT combined scores (p=0.02).

Interestingly, results indicate a significant role for subjective cognitive complaints and potential exaggeration of cognitive symptoms post-COVID-19 infection, in the absence of differences in objective performance in most cognitive domains. While subtle differences are seen on some executive function measures, mean group differences are small, and in the context of higher SIMS subscale scores, may not be readily interpretable. Studies employing similarly comprehensive neuropsychological assessments including validity measures in larger samples are needed to further disambiguate potential objective cognitive performance decrements from subjectively experienced difficulties.

Baseline assessment of cognitive performance is common practice under many concussion management protocols and is required for collegiate athletes by the NCAA. The purpose of baseline cognitive assessment is to understand an athlete’s individual uninjured cognitive performance, as opposed to using population normative data. This baseline can then serve as a reference point for recovery after concussion and can inform return-to-play decisions. However, multiple factors, including lack of effort, can contribute to misrepresentation of baseline results which raises concern for reliability during return-to-play decision-making. Measuring effort across a continuum, rather than as a dichotomous variable (good versus poor effort) may provide informative insight related to cognitive performance at baseline.

Collegiate athletes (n = 231) completed the Immediate Post-Concussion Assessment and Cognitive Test (ImPACT) as part of their baseline pre-participation concussion evaluation. ImPACT creates composite scores of Verbal Memory, Visual Memory, Visual-Motor Speed, and Reaction Time. Baseline self-reported symptoms and total hours of sleep the night prior to testing are also collected through ImPACT. ImPACT has one embedded indicator within the program to assess effort, and research has identified an additional three embedded indicators. Athletes were also administered one stand-alone performance validity test, either the Medical Symptom Validity Test (n = 130) or the Rey Dot Counting Test (n = 101), to independently measure effort. Effort was estimated across a continuum (zero, one, two, or three or more failed effort indicators) with both stand-alone and embedded effort indicators. We evaluated the relationship between effort, symptoms, self-reported sleep, Reaction Time composite score and Visual-Motor Speed composite score using a linear regression model.

We found that 121 athletes passed all effort indicators, while 39 athletes failed only one effort indicator, 40 athletes failed two effort indicators, and 31 athletes failed three or four (three+) effort indicators. Self-reported symptoms and total hours of sleep were not related to effort, but Reaction Time and VisualMotor Speed composites were. Specifically, performance on the Visual-Motor Speed composite was significantly worse for athletes who failed two or three+ effort indicators compared to athletes who did not fail any, and performance on the Reaction Time composite was significantly worse only for athletes who failed three+ effort indicators. Additionally, athletes who failed one or more effort indicators and reported less sleep performed worse on both the Visual-Motor Speed and Reaction Time composites, compared to those who reported less sleep and did not fail any effort indicators.

Athletes who failed one effort indicator did not perform significantly worse on Reaction Time and Visual-Motor Speed composites compared to those who passed all effort indicators. However, 31% of athletes failed two or more effort indicators and these athletes performed worse on cognitive tests, likely due to factors impacting their ability to put forth good effort. These results suggest that effort is more complex than a previously used dichotomous variable and highlights the importance of using several indicators of effort throughout baseline assessments. In addition, the importance of sleep should be emphasized during baseline assessments, especially when effort is questionable.

Accurate identification of Attention-Deficit/Hyperactivity Disorder (ADHD) is complicated by possible secondary gain, overlap of symptoms with psychiatric disorders, and face validity of measures (Suhr et al., 2011; Shura et al., 2017). To assist with diagnostic clarification, an experimental Dissimulation ADHD scale (Ds-ADHD; Robinson & Rogers, 2018) on the MMPI-2 was found to distinguish credible from non-credible respondents defined by Performance Validity Test (PVT)-based group assignment in Veterans (Burley et al., 2023). However, symptom and performance validity have been understood as unique constructs (Van Dyke et al., 2013), with Symptom Validity Tests (SVTs) more accurately identifying over-reporting of symptoms in ADHD (White et al., 2022). The current study sought to evaluate the effectiveness of the Ds-ADHD scale using an SVT, namely the Infrequency Index of CAARS (CII; Suhr et al., 2011), for group assignment within a mixed sample of Veterans.

In this retrospective study, 187 Veterans (Mage = 36.76, SDage = 11.25, Medu = 14.02, SDedu = 2.10, 83% male, 19% black, 78% white) were referred for neuropsychological evaluation of ADHD and administered a battery that included internally consistent MMPI-2 and CAARS profiles. Veterans were assigned to a credible group (n=134) if CII was <21 or a non-credible group (n=53) if CII was >21. The Ds-ADHD scale was calculated for the MMPI-2. Consistent with Robinson and Rogers (2018), “true” answers (i.e., erroneous stereotypes) were coded as 1 and “false” answers were coded as 2, creating a 10- to 20-point scale. Lower scores were associated with a higher likelihood of a feigned ADHD presentation.

Analyses revealed no significant differences in age, education, race, or gender (ps > .05) between credible and non-credible groups. An ANOVA indicated a significant difference between groups (F[1,185] = 24.78, p <.001; Cohen’s d = 0.80) for Ds-ADHD raw scores. Veterans in the non-credible group reported more “erroneous stereotypes” of ADHD (M raw score = 13.23, SD = 2.10) than those in the credible group (M = 14.94, SD = 2.13). A ROC analysis indicated AUC of .72 (95% CI = .64 to .80). In addition, a Ds-ADHD cut score of <12 resulted in specificity of 94.5% and sensitivity of 22.6%, whereas a cut score of <13 resulted in specificity of 85.8% and sensitivity of 50.9%. When analyzing other CII cut scores recommended in the literature, results were essentially similar. Specifically, analyses were repeated when group assignment was defined by cut score of CII<18 and by removing an intermediate group (CII = 18 to 21; n=24).

The Ds-ADHD scale demonstrated significant differences between credible and non-credible respondents in a Veteran population. Results suggest a cut score of <12 had adequate specificity (.95) with low sensitivity (.23). This is consistent with findings using PVTs for group assignment that indicated a cut score of <12 had adequate specificity (.92) with low sensitivity (.19; Burley et al., 2023). Taken together, findings suggest that the Ds-ADHD scale demonstrates utility in the dissociation of credible from non-credible responding. Further research should evaluate the utility of the scale in other clinical populations.

The MMPI-2-RF contains scales that assess different types of invalid response styles, especially potential symptom over-reporting (e.g., F-r, Fs, Fp-r, FBS-r, RBS). However, these scales are not designed to specifically capture noncredible symptoms reports associated with Attention-Deficit/Hyperactivity Disorder (ADHD). Robinson & Rogers (2018) proposed the experimental Dissimulation ADHD validity scale (Ds-ADHD) on the MMPI-2-RF that was effective in distinguishing credible and non-credible ADHD diagnoses via a simulator-based study. Within the current study, the Ds-ADHD scale was compared to the established MMPI-2-RF validity scales within a mixed sample of U.S. Military Veterans.

173 Veterans (Mage = 36.18, SDage = 11.10, Medu = 14.01, SDedu = 2.11, 88% male, 81% White, 17% Black) completed a neuropsychological evaluation which included an internally consistent MMPI-2-RF profile and up to 10 performance validity tests (PVTs) as well as a question about a possible ADHD diagnosis. The credible group was determined if participants passed all PVTs (n=146) and completed at least 2 PVTs. The non-credible group was determined by failing two or more PVTs (n=27). Group assignment was clinically confirmed. The Ds-ADHD scale was calculated according to Robinson & Rogers’ (2018); responses of “true” (i.e., erroneous stereotypes) were coded as 1 and “false” answers were coded 2, creating a 10- to 20-point scale. Thus, lower scores would be associated with a higher likelihood of a feigned ADHD presentation. Other MMPI-2-RF validity scales of interest included F-r, Fs, Fp-r, FBS-r, and RBS.

The established MMPI-2-RF validity scales were significantly correlated with PVT group membership, but correlations were weak to moderately strong (rS ranged from -.43 to -.18; ps < .05). A series of stepwise regression models were completed with the Ds-ADHD scale and one of the MMPI-2-RF validity scales as independent variables, with group membership as the dependent variable. Ds-ADHD) contributed uniquely to each model (CÜ ranged from .03 to .04, ps < .05). The established MMPI-2-RF validity scales effectively classified group membership (AUC values ranged from .57 to .68), and the Ds-ADHD scale had a marginally higher AUC (.69); however, it was not statistically significantly stronger than any of the established scales (ps > .05).

Clinicians interested in identifying potentially simulated ADHD presentations with the MMPI-2-RF may desire to calculate the Ds-ADHD scale, which previously only had support from a simulator-based study. The Ds-ADHD scale significantly contributed to each model, suggesting that it helped explain groups over and above each of the traditional MMPI-2-RF validity scales. However, it only had a marginally stronger ability to classify participants, indicating that there may be diminishing returns for clinicians. Among the traditional validity scales, RBS and F-r best classified groups, and FBS-r was the least effective. This study employed a cross-sectional design in a mixed sample of Veterans undergoing a neuropsychological evaluation. Future research should focus on replicating the findings using a credible sample that was limited to an independently verified diagnosis of ADHD.

Ensuring test-taking validity is a crucial part of any neuropsychological evaluation. While all batteries ought to include well established test-taking validity measures regardless, it can still be helpful to be aware of an increased chance of poor performance validity prior to initiating testing. Studies repeatedly demonstrate that it is very difficult to predict which patient, particularly those without any clear incentive for poor test performance, will have invalid test performances based purely on subjective clinical judgment. Therefore, there is a need for an objective predictor of poor test taking validity. This study examines if a high endorsement of cognitive symptoms can indicate likely failure on test-taking validity measures.

All patients at an outpatient neurological clinic completed an intake background form prior to testing. On this form, patients were asked to endorse in which, if any, of nine cognitive areas they may be experiencing difficulty (memory, attention/concentration, word finding, etc.). Patients who endorsed at least eight out of the nine clinical symptoms on the intake form were included in the current study (N=7; age range 36-43 years). All patients were clinically referred for a comprehensive neuropsychological evaluation with a variety of conditions (e.g., stroke, memory concerns, and post-COVID-19 syndrome). Importantly, none of these patients were referred within a forensic context, and therefore, they did not have any clear external motivation or secondary gain. In addition to a battery of individual neuropsychological measures, each patient was administered performance validity tests (Test of Memory Malingering, Reliable Digits, and CVLT-3 Forced Choice).

In this sample, 57% of patients who endorsed all - or nearly all - cognitive symptoms on an intake form failed test-taking validity measures. Patients who failed validity measures did not meet passing criteria on two or more embedded or independent performance validity tests. This signifies a much higher rate than the typically observed base rates (∼15%) of test-taking invalidity across non-forensic clinical settings.

Preliminary findings suggest that those who indicate having cognitive problems in all (or nearly all) listed domains fail validity measures at a higher than expected rate, supporting the use of responses on a background from to indicate likely poor performance validity. Identification of high rates of symptomatic complaints, particularly symptoms that may extend beyond the initial referral question, should prompt practitioners to keenly evaluate performance validity and consider the results within the context of the patient’s presentation.

Inconsistent relationships between subjective and objective performance have been found across various clinical groups. Discrepancies in these relationships across studies have been attributed to various factors such as patient characteristics (e.g., level of insight associated with cognitive impairment) and test characteristics (e.g., using too few measures to assess different cognitive domains). Although performance and symptom invalidity are common in clinical and research settings and have the potential to impact responding on testing, previous studies have not explored the role of performance and symptom invalidity on relationships between objective and subjective performance. Therefore, the current study examined the impact of invalidity on performance and symptom validity tests (PVTs and SVTs, respectively) on the relationship between subjective and objective cognitive functioning.

Data were obtained from 299 Veterans (77.6% male, mean age of 48.8 years (SD = 13.5)) assessed in a VA medical center epilepsy monitoring unit from 2008-2018. Participants completed a measure of subjective functioning (i.e., the Patient Competency Rating Scale), PVTs (i.e., Word Memory Test, Test of Memory Malingering, Reliable Digit Span), SVTs (i.e., Minnesota Multiphasic Personality Inventory-2-Restructured Form Response Bias Scale, Structured Inventory of Malingered Symptomatology), and neuropsychological measures assessing objective cognitive performance (e.g., Trail Making Test parts A and B). Pearson correlations were conducted between subjective functioning and objective cognitive performance in the following groups: 1.) PVT and SVT valid, 2.) PVT and SVT invalid, 3.) PVT-only invalid, 4.) SVT-only invalid. Using Fisher’s r-to-z transformation, tests for the differences between correlation coefficients were then conducted between the PVT and SVT valid vs. PVT and SVT invalid groups, and the PVT-only invalid vs. SVT-only invalid groups.

Participants with fully valid PVT and SVT performances demonstrated generally stronger relationships between subjective and objective scores (r’s = .058 - .310) compared to participants with both invalid PVT and SVT scores (r’s = -.033 - .132). However, the only significant difference in the strengths of correlations between the groups was found on Trail Making Test Part B (p = .034). In separate exploratory analyses due to low group size, those with invalid PVT scores only (fully valid SVT) demonstrated generally stronger relationships between subjective and objective scores (r’s = -.101 - .741) compared to participants with invalid SVT scores only (fully valid PVT; r’s = -.088 - .024). However, the only significant difference in the strengths of correlations between the groups was found on Trail Making Test Part A (p = .028).

The present study suggests that at least some of the discrepancies in previous studies between subjective and objective cognitive performance may be related to performance and symptom validity. Specifically, very weak relationships between objective and subjective performance were found in participants who only failed SVTs, whereas relationships were stronger in those who only failed PVTs. Therefore, findings suggest that including measures of PVTs and SVTs in future studies investigating relationships between subjective and objective cognitive performance is critical to ensuring accuracy of conclusions that are drawn.

Previous literature has studied the cognitive processes that contribute to performance on the Stroop interference condition in adults and found that the Stroop task performance (i.e., color-word interference) is comprised of multiple cognitive skills, including speed of visual search, working memory, and conflict monitoring (Perianez et al. 2020). However, the relationship of these cognitive processes to Stroop interference in youth remains understudied. Moreover, no studies have examined the contribution of effort measurement to the interference condition in healthy youth.

Golden Stroop Test interference performance was examined in healthy youth athletes (n=174) aged 8-16 years (mean age=12.07) who participated in a baseline neuropsychological evaluation as part of a clinical research program on sports concussion. Predictor variables included speed of visual search, working memory, processing speed, verbal fluency effort (i.e., validity tests), visuospatial abilities, visual processing, and executive functioning skills such as cognitive flexibility and reasoning.

Speed of visual search as measured by Trail Making Test visual scanning time (p<0.00), and effort as measured by Reliable Digit Span and Trail Making Test ratio (p=0.03; p<0.00, respectively) significantly contributed to Stroop interference performance in healthy youth. We provided three validity measures; however, only those requiring higher-order cognitive processes predicted Stroop performance: Reliable Digit Span (p=0.03) and the Trail Making Test ratio (p<0.00). The standalone validity measure (TOMM) was not a significant predictor of Stroop performance (p>0.05).

In contrast to adults, working memory and processing speed did not significantly predict Stroop performance, while visual search speed did predict Stroop interference. Furthermore, two embedded validity indicator (EVI) measures predicted Stroop interference, in contrast to a standalone validity measure requiring lower cognitive processes, which did not predict Stroop performance. Therefore, EVI’s that include an executive functioning component may not accurately represent effort in youth, perhaps due to their less developed executive functioning relative to adults (Lezak et al., 2012; Shanmugan & Satterthwaite, 2017). Overall, understanding the cognitive processes contributing to Stroop performance in healthy youth will allow clinicians to better detect deficits in those cognitive processes and understand how they may impact Stroop performance. This would lead to a better understanding of executive functioning and the accurate measurement of effort in healthy youth.

This study examined false positive rates of performance validity test (PVT) failure in a group of monolingual (English-speaking) White non-Hispanic/Latinx (non-HL), monolingual (English-speaking) Hispanic/Latinx (HL), and bilingual (English- and Spanish-speaking) HL patients evaluated at an academic medical center. Research on classification accuracy of embedded performance validity tests (PVTs) is limited in HL and bilingual populations. Cultural test biases or language differences could inaccurately cause scores below PVT cutoff levels.

The project involved secondary analysis of a deidentified dataset (N=391). Participants were included if they were between the ages of 18 and 64, had data from the initial visit, had an IQ greater than or equal to 70, were not diagnosed with dementia or major or mild cognitive impairment, and identified as either White non-HL or HL. Participants were required to have completed at least two PVTs. Participants who were not administered the Test of Memory Malingering (TOMM; n=95) or who scored below a highly sensitive Trial 1 cutoff (<46; n=86) were excluded. The final sample included 210 participants, which included monolingual non-HL participants (n=114), monolingual HL participants (n=44), and bilingual HL participants (n=52). Failure rates on eight PVTs were examined by participant group: Reliable Digit Span (RDS), Auditory Verbal Learning Test Recognition (AVLT), Logical Memory Recognition (LM), Visual Reproduction Recognition (VR), Trail Making Test Ratio (TMT rat), Rey Complex Figure Test Recognition (RCFT), Semantic Word Generation (animals; SWG), and Finger Tapping (TAP).

Groups were not significantly different in age. Monolingual non-HL participants had completed more years of education than monolingual and bilingual HL groups (13.7, 12.7, and 12.8 years respectively). In the whole sample, 8.6% (n=18) failed two or more PVTs. In the monolingual non-HL group, 8.8% (n=10) failed two or more PVTs, while 9.1% (n=4) of the monolingual HL group and 7.7% (n=4) of the bilingual HL group failed two or more PVTs (n.s.). Within the monolingual non-HL test set, failure rates were above 10% on SWG (12.73%) and TAP (17.7%). Failure rates above 10% in the monolingual HL set were found on SWG (11.6%). Failure rates above 10% in bilingual HL measures were observed on SWG (13.5%) and TAP (10.8%).

Total PVT failure rates did not significantly differ between groups. Across groups, performance was above a common false positive threshold of 10% on SWG. Monolingual non-HL and bilingual participants also had elevated failure rates on TAP.

There is a dearth of an appropriate standardized tool to assess neuropsychological functions in rural population, which has low literacy rates, are culturally diverse, and have limited access to healthcare resources. The NIREH Neuropsychological Battery for Rural Population (NINB-RP) is a relatively brief and easy-to-administer battery comprising multiple tests that are modified or adopted as per rural community settings to evaluate verbal learning, fine coordination, attention efficiency, executive task, concentration, and visual attention, mental flexibility, and motor coordination in rural populations. The present study aimed to examine the clinical validity and establish cut-off scores for impairment of neuropsychological functions for different age, gender, and education levels of NINB-RP in a rural community in central India.

This was a prospective cross-sectional study conducted in participants aged > 18 years (n=2952, M: F=1407:1545) recruited through a stratified sampling technique from 23 randomly selected villages from central India. The data of nine neuropsychological tests [(Finger and Tweezer dexterity test (FDT, TDT); Digit Forward and Backward test (DFT, DBT); Serial subtraction test (SST); Trail Making-A and B; Finger Tapping test (FTT); and Letter Digit Substitution test, LDST)] from 215 cognitively impaired and 2737 healthy control subjects were analyzed. The tests were performed in a village school/community hall or an outdoor camp. Independent sample t-test, Chi-square test, and Receiver Operating Characteristic (ROC) curve were used to calculate the area under the curve (AUC), cut-off scores, and sensitivity (ST)/specificity (SP) values for seven conditions, i.e., gender (male vs. female), age groups (up to 49 years and above 50 years); and educational levels (illiterate, intermediate and college). For those variables where ST/SP values were lower than 0.70, a unique cut-off score was calculated for the entire sample, adjusting by age and educational levels.

A significant difference in mean (median) scores between the healthy control and cognitively impaired groups were observed in all tests except Trail Making A and B and LDST. The AUC for most of the tests ranged from 0.70 to 0.81, and the ST/SP values ranged from 69-73% and 65-75%, respectively. The results showed that most tests of NINB-RP reached moderate to good sensitivity and specificity for gender, age and education levels, except for DBT for females, above 50 years, and illiterate and intermediate education groups. FDT for males [AUC: 0.85 (95%CI0.80-0.91], ST/SP=76/82%] and females [(AUC=0.78 (95%CI0.74-0.82), ST/SP=71/70%], TDT for intermediate education group [AUC=0.82 (95%CI0.60-1.00), ST/SP=86/83%] and FTT for less than 49 years age group [AUC=0.75 (95%CI0.67-0.84), ST/SP=71/76%] were the most useful tests to discriminate among healthy control and cognitively impaired rural population.

The present study is an attempt to establish the cut-off scores of a neuropsychological battery for a large rural population in the community setting. The proposed cut-off values might be helpful in clinical assessment in rural areas where clinical neuropsychology services are not readily available. NINB-RP can be a valuable tool for clinical research studies in rural communities. Further studies on similar samples in other countries need to be undertaken.

People with psychotic disorders often experience neurocognitive deficits, such as neurocognitive impairment (NCI), which can negatively affect their daily activities (e.g., performing independent tasks) and recovery. Because of this, the American Psychology Association advocates integrating neurocognitive testing into routine care for people living with psychotic disorders, especially those in their first episode, to inform treatment and improve clinical outcomes. However, in low-and-middle income countries (LMICs), such as Uganda where the current study took place, administering neurocognitive tests in healthcare settings presents numerous challenges. In Uganda there are few resources (e.g., trained clinical staff, and culturally relevant and normed tests) to routinely offer testing in healthcare settings. NeuroScreen is a brief, highly automated, tablet-based neurocognitive testing tool that can be administered by all levels of healthcare staff and has been translated into indigenous Ugandan languages. To examine the psychometric properties of NeuroScreen, we measured convergent and criterion validity of the NeuroScreen tests by comparing performance on them to performance on a traditional battery of neurocognitive tests widely used to assess neurocognition in people with psychotic disorders, the Matric Consensus Cognitive Battery (MCCB).

Sixty-five patients admitted into Butabika Mental Referral Hospital in Uganda after experiencing a psychotic episode and forty-seven demographically similar control participants completed two neurocognitive test batteries: the MCCB and NeuroScreen. Both batteries include tests measuring the neurocognitive domains of executive functioning, working memory, verbal learning, and processing speed. Prior to completing each battery, patients were medically stabilized and could not exhibit any positive symptoms on the day of testing. On the day of testing, medication dosages were scheduled so that patients would not experience sedative effects while testing. To examine convergent validity, we examined correlations between overall performance on NeuroScreen and the MCCB, as well as tests that measured the same neurocognitive domains. To examine criterion validity, an ROC curve was computed to examine the sensitivity and specificity of NeuroScreen to detect NCI as defined by the MCCB.

There was a large correlation between overall performance on NeuroScreen and the MCCB battery of tests, r(110) = .65, p < .001. Correlations of various strengths were found among tests measuring the same neurocognitive domains in each battery: executive functioning [r(110) = .56 p <.001], processing speed [r(110) = .44, p <.001)], working memory [r(110) = .29, p<.01], and verbal learning [r(110) = .22, p < .01]. ROC analysis of the ability of NeuroScreen to detect MCCB defined NCI showed an area under curve of .798 and optimal sensitivity and specificity of 83% and 60%, respectively.

Overall test performance between the NeuroScreen and MCCB test batteries was similar in this sample of Ugandans with and without a psychotic disorder, with the strongest correlations in tests of executive functioning and processing speed. ROC analysis provided criterion validity evidence of NeuroScreen to detect MCCB defined NCI. These results provide support for use of NeuroScreen to assess neurocognitive functioning among patients with psychotic disorders in Uganda, however more work needs to be to determine how well it can be implemented in this setting. Future directions include assessing cultural acceptability of NeuroScreen and generating normative data from a larger population of Ugandan test-takers.

Performance validity testing (PVT) is important in neuropsychological evaluations to ensure accurate interpretation of performance. While research shows children pass PVTs with adult cut-offs, PVTs are more commonly used with adults (Lippa, 2018). The Test of Memory Malingering (TOMM), a standalone PVT, is commonly used with adults and children (DeRight & Carone, 2015). The Reliable Digit Span (RDS), an embedded PVT derived from the Digit Span subtest of the Wechsler Intelligence Scales (Wechsler Intelligence Scale for Children-4th Edition, WISC-IV; Wechsler, 2003), is less commonly used with children (DeRight & Carone, 2015). RDS cut-off scores are associated with an increased rate of false positives in children, indicating mixed results regarding the clinical utility in pediatric populations (Welsh et al., 2012). Research shows that youth with a history of concussion (HOC) may demonstrate suboptimal effort for many reasons (e.g., external incentives, boredom, pressure), thus highlighting the need to investigate the utility of PVTs in this population (Araujo et al., 2014; DeRight & Carone, 2015). The present study aimed to examine the clinical utility of RDS in detecting poor effort on the TOMM in youth athletes with a HOC.

Participants included 174 youth athletes aged 8 to 18 (20.1% female; 42.5% people of color (POC)) who completed a baseline neuropsychological battery that included the TOMM and WISC-IV Digit Span. Of the sample, 29 youth athletes reported a HOC (13.8% female; 37.9 POC). RDS was calculated for each Digit Span administration, and sensitivity (SN) and specificity (SP) were calculated for RDS when invalid performance was operationalized by a more stringent cut-off score of <49 to increase the SN of the TOMM Trial 1 (Stenclik et al., 2013). Receiver operator characteristics (ROC) curve analysis determined whether RDS performance accurately predicted participants’ performance on the TOMM.

The ROC curve analysis resulted in an area under the curve (AUC) of just 0.427 for RDS. A cut-off score of <7 (as suggested by Kirkwood et al. (2011)) for RDS results in 100% SN, 8.3% SP, 5% positive predictive validity (PPV), and 95% negative predictive validity (NPV). However, a cut-off score of <9 for RDS results in 75% SN, 15% SP, 25% PPV, and 75% NPV.

Little research shows the utility of different PVTs predicting children’s performance on other PVTs, despite evidence that children with a HOC are vulnerable to variable or insufficient effort (Araujo et al., 2014; DeRight & Carone, 2015). In a sample of 29 youth athletes with a HOC, RDS predicted TOMM performance at rates worse than chance. While RDS has advantages as an embedded PVT, its limited ability to predict performance on a standalone PVT suggests interpreting with great caution. These findings highlight the importance of implementing multiple PVTs throughout testing to ensure accurate findings and interpretations, particularly in youth with a HOC. The small sample size is a limitation that possibly impacted the ability of RDS to predict TOMM performance. Further research is needed to understand the utility of RDS as a predictor of PVT performance in different populations. Replication of these findings with a larger sample size is needed to provide confirmatory evidence of poor predictive performance of the RDS.

Previous investigations have demonstrated the clinical utility of the Delis-Kaplan Executive Function System (D-KEFS) Color Word Interference Test (CWIT) as an embedded validity indicator in mixed clinical samples and traumatic brain injury. The present study sought to cross-validate previously identified indicators and cutoffs in a sample of adults referred for psychoeducational testing.

Archival data from 267 students and community members self-referred for a psychoeducational evaluation at a university clinic in the South were analyzed. Referrals included assessment for attention-deficit hyperactivity disorder, specific learning disorder, autism spectrum disorder, or other disorders (e.g., anxiety, depression). Individuals were administered subtests of the D-KEFS including the CWIT and several standalone and embedded performance validity indicators as part of the evaluation. Criterion measures included The b Test, Victoria Symptom Validity Test, Medical Symptom Validity Test, Dot Counting Test, and Reliable Digit Span. Individuals who failed 0 criterion measures were included in the credible group (n = 164) and individuals failing 2 or more criterion measures were included in the non-credible group (n = 31). Because a subset of the sample were seeking external incentives (e.g., accommodations), individuals who failed only 1 of the criterion measures were excluded (n = 72). Indicators of interest included all test conditions examined separately, the inverted Stroop index (i.e., better performance on the interference trial than the word reading or color naming trials), inhibition and inhibition/switching composite, and sum of all conditions.

Receiver Operating Characteristics (ROC) curves were significant for all four conditions (p < .001) and the inverted stroop index (p = .032). However, only conditions 2, 3 and 4 met minimal acceptable classification accuracy (AUC = .72 - 81). ROC curves with composite indicators were also significant (p < .001), with all three composite indicators meeting minimal acceptable classification accuracy (AUC = .71- .80). At the previously identified cutoff of age corrected scale score of 6 for all four conditions, specificity was high (.88 -.91), with varying sensitivity (.23 - .45). At the previously identified cutoff of .75 for the inverted stroop index, specificity was high (.87) while sensitivity was low (.19). Composite indicators yielded high specificity (.88 - .99) at previously established cutoffs with sensitivity varying from low to moderate (.19 - .48). Increasing the cutoffs (i.e., requiring higher age corrected scale score to pass) for composite indicators increased sensitivity while still maintaining high specificity. For example, increasing the total score cutoff from 18 to 28 resulted in moderate sensitivity (.26 vs .52) with specificity of .91.

While a cutoff of 6 resulted in high specificity for most conditions, the sum of all four conditions exhibited the strongest classification accuracy and appears to be the most robust indicator which is consistent with previous research (Eglit et al., 2019). However, a cutoff of 28 as opposed to 18 may be most appropriate for psychoeducational samples. Overall, the results suggest that the D-KEFS CWIT can function as a measure of performance validity in addition to a measure of processing speed/executive functioning.