The Brief Visuospatial Memory Test-Revised (BVMT-R) Recognition Discrimination (RD) index has emerged as an embedded performance validity test (PVT). However, there do not appear to be any studies that have examined its utility in Spanish-speaking samples. This pilot study examined the classification accuracy of the BVMT-R RD for detecting performance invalidity in a Spanish-speaking forensic sample.

This cross-sectional study utilized a sample of 89 Spanish speakers that were administered the BVMT-R during an outpatient neuropsychological evaluation. Out of the 89 Spanish speakers, 43 were subjects in litigation, 32 were neurological patients evaluated for clinical purposes, and 14 were healthy controls. The sample was 67% male/33% female, 53% South American, 33% Caribbean (Dominican, Puerto Rican, Cuban), 10% Central American, 3% North American (Mexican), and 1% Spanish, with a mean age of 44.2 years (SD = 14.2; range = 20-78) and mean education of 11 years (SD = 3.7; range = 0-20). Test administration for each patient was completed in Spanish by a fluent, Spanish-speaking examiner. In total, 64/89 (72%) were classified as valid and 25/89 (28%) as invalid based on performance across the Test of Memory Malingering (TOMM), at least one additional PVT (Rey-15 item memory test; Rey Dot Counting Test; Reliable Digit Span; WHO-AVLT recognition trial) and objective diagnostic criteria identifying invalid performance. Analyses included three univariate analyses of variance (ANOVA), with the groups (healthy vs neurological vs litigation) as independent variables and performance on BVMT-RD as the dependent variable.

Statistically significant differences among the groups were found F(2,86)=8.32, p < .001). Post-hoc analysis (Scheffe test) showed the mean of the litigation group to be significantly lower than the means of the other two groups (healthy and neurological), which showed no difference between them. An ANOVA with validity groups as the fixed factor and BVMT-R RD index as the dependent variable was significant F(1,85)= 21.02, p <.001). Results of a ROC curve analysis yielded statistically significant AUC (.794). The optimal cut-score was BVMT-R RD < 5 (48% sensitivity/88% specificity).

Results of the BVMT-R RD index in this Spanish-speaking population differed by subgroup, with worse performance seen in individuals involved in litigation, compared to those who were not (healthy and neurological). Notably, the BVMT-R RD index significantly differentiated validity groups, maintaining adequate sensitivity and good specificity. Overall, results demonstrate promise for BVMT-RD as a PVT for Spanish-speaking populations.

Performance validity tests (PVTs) provide a methodological approach to detecting credible neurocognitive performances. This proves invaluable to the diagnostic process, as it allows neuropsychologists to objectively determine if an evaluation reflects a patient’s true neurocognitive abilities or if external factors are impacting the results. However, their addition to a testing battery can increase an already lengthy evaluation. As such, there is a need for sensitive but less time intensive PVTs. The purpose of this study is to validate the Coin-in-Hand (CIH) procedure as a quick and effective PVT within a veteran population.

68 English-speaking patients were identified from an outpatient neuropsychological assessment dataset. Performances were correlated to the well- validated Reliable Digit Span (RDS), and several other soft indicators of task engagement including expanded COWAT, BVMT-False Alarms (FA), WCST Failure to Maintain Set (FTM), TOMM, and the RBANS Effort Index (EI). All participants attempted CIH and RDS, testing was discontinued if 2 or more PVTs were invalid. An AUC analysis was conducted to determine how well the CIH discriminated between valid and invalid performance and determine the tests optimal cut-off score (sensitivity > 0.90 while maintaining the highest possible specificity). Logistic Regression was conducted to determine how well the CIH predicted performance validity.

Subject mean(SD) age and education were 55.25 (16.06) and 13.41 (2.55) years, respectively. 17% female, 60% Caucasian, and 32% Black. Descriptive statistics for each of the other performance validity tests were gathered. The CIH demonstrated low diagnostic accuracy (AUC = .66; p >.05; CI = .51 -.81); a cut score of <8 resulted in a sensitivity of .96 and a specificty of .64. Logistic Regression showed that CIH performance significantly predicted performance validity (X2 = -0.93; df = 1; N = 68; p < .05), accounting for 18-28% of the variance in performance classification (Cox & Snell R2 = .18; Nagelkerke R2 = .28). It correctly classified 96% of valid performers, but only correctly classified 35% of invalid performers, with an overall correct prediction rate of 83%. A predicted chase in log odds (B= -.93) and odd ratio [Exp (B) =.40] indicated that every unit increase in CIH score was associated with a decrease probability of performance invalidity. Logistic regression was also used to calculate the probability of performance invalidity at each possible CIH score (Table 1).

Results suggests that poor performance on CIH does not necessarily equate to invalid performances, but instead, should act as a screener to cue neuropsychologists working with Veterans that additional PVTs should be considered. Overall, it was determined that CIH was able to correctly predict 35% of invalid performers and 96% of valid performers, with an overall correct prediction rate of 83%, suggesting the procedure may be too simple to be an effective standalone PVT for clinical use. These results also highlight that every correct response on the CIH was associated with a decreased probability of performance invalidity. Additionally, an AUC analysis determined the tests optimal cut off score to be <8, suggesting that shortening the procedure may be as effective as giving the full 10 trials.

The prevalence of ADHD diagnoses more than doubled in VA settings between 2009 and 2016 (Hale et al., 2020). However, attentional difficulties are not exclusive to ADHD and can also be seen in non-neurodevelopmental disorders, including depression, anxiety, substance use, and PTSD (Marshall et al., 2018, Suhr et al., 2008). Further, patients can easily feign symptoms of ADHD with few available instruments for accurate detection (Robinson & Rogers, 2018). Given the significant symptom overlap and rising rates of reported ADHD among Veterans, accurate detection of feigned ADHD is essential.

This study examined the utility of the experimental Dissimulation ADHD scale (Ds-ADHD; Robinson & Rogers, 2018) on the MMPI-2, in detecting feigned ADHD presentation within a mixed sample of Veterans.

In this retrospective study, 173 Veterans (Mage = 36.18, SDage = 11.10, Medu = 14.01, SDedu = 2.11, 88% male, 81% White, and 17% Black) were referred for neuropsychological evaluation of ADHD that included the MMPI-2 and up to 10 PVTs. Participants were assigned to a credible group (n=146) if they passed all PVTs or a non-credible group (n=27) if they failed two or more PVTs. Group assignment was also clinically confirmed. The Ds-ADHD was used to differentiate groups who either had credible or non-credible performance on cognitive measures. Consistent with Robinson and Rogers’ study, “true” answers (i.e., erroneous stereotypes) were coded as 1 and “false” answers were coded as 2, creating a 10- to 20-point scale. Lower scores were associated with a higher likelihood of a feigned ADHD presentation.

Preliminary analyses revealed no significant group differences in age, education, race, or gender (ps > .05). An ANOVA indicated a significant difference between groups (F[1, 171] = 10.44, p = .001; Cohen’s d = .68) for Ds-ADHD raw scores; Veterans in the non-credible group reported more “erroneous stereotypes” of ADHD (M raw score = 13.33, SD = 2.20) than those in the credible group (M = 14.82, SD = 2.20). A ROC analysis indicated AUC of .691 (95% CI = .58 to .80). In addition, a cut score of <12 resulted in specificity of 91.8% and sensitivity of 18.5%, whereas a cut score of <13 resulted in specificity of 83.6% and sensitivity of 44.4%.

The Ds-ADHD scale demonstrated significant differences between credible and non-credible respondents in a real-world setting. Previously, this scale has primarily been studied within laboratory settings. Further, results indicate a cut score of <12 could be used in order to achieve adequate specificity (i.e., >90%), which were similar findings to a study examining SVT-based groups (Winiarski et al., 2023). These results differ slightly from prior research by Robinson and Rogers (2018), who indicated a cut score of <13 based on the initial simulation-based study. In similar clinical settings, where there are high rates of psychiatric comorbidity, a cut score of <12 may prove clinically useful. However, this cut-score was associated with low sensitivity within this mixed Veteran sample. Further research should focus on replicating findings within other clinical settings, including ones with larger non-credible samples.

Assessment of performance validity during neuropsychology evaluation is essential to reliably interpret cognitive test scores. Studies (Webber et al., 2018; Wisdom, et al., 2012) have validated the use of abbreviated measures, such as Trial 1 (T1) of the Test of Memory Malingering (TOMM), to detect invalid performance. Only one study (Bauer et al., 2007) known to these authors has examined the utility of Green’s Word Memory Test (WMT) immediate recall (IR) as a screening tool for invalid performance. This study explores WMT IR as an independent indicator of performance validity in a mild TBI (mTBI) veteran population.

Participants included 211 (Mage = 32.1, SD = 7.4; Medu = 13.1, SD = 1.64; 94.8% male; 67.8% White) OEF/OIF/OND veterans with a history of mTBI who participated in a comprehensive neuropsychological evaluation at one of five participating VA Medical Centers. Performance validity was assessed using validated cut scores from the following measures: WMT IR and delayed recall (DR); TOMM T1; WAIS-IV reliable digit span; CVLT-II forced choice raw score; Wisconsin Card Sorting Test failure to maintain set; and the Rey Memory for Fifteen Items test, combo score. Sensitivity and specificity were calculated for each IR score compared with failure on DR. In addition, sensitivity and specificity were calculated for each WMT IR score compared to failure of at least one additional performance validity measure (excluding DR), two or more measures, and three or more measures, respectively.

Results indicated that 46.8% participants failed to meet cut offs for adequate performance validity based on the standard WMT IR cut score (i.e., 82.5%; M = 81.8%, SD = 17.7%); however, 50.2% participants failed to meet criteria based on the standard WMT DR cut score (M = 79.8% SD = 18.6%). A cut score of 82.5% or below on WMT IR correctly identified 82.4% (i.e., sensitivity) of subjects who performed below cut score on DR, with a specificity of 94.2%. Examination of IR cutoffs compared to failure of one or more other PVTs revealed that the standardized cut score of 82.5% or below had a sensitivity of 78.2% and a specificity of 72.4%; whereas, a cut score of 65% or below had a sensitivity of 41% and a specificity of 91.3%. Similarly, examination of IR cutoffs compared to failure of two or more additional PVTs revealed that the cut score of 60% or below had a sensitivity of 45.7% and specificity of 93.1% ; whereas, a cut score of 57.5% or below had a sensitivity of 57.9% and specificity of 90.9% when using failure of three or more PVTs as the criterion.

Results indicated that a cut score of 82.5% or below on WMT IR may be sufficient to detect invalid performance when considering WMT DR as criterion. Furthermore, WMT IR alone, with adjustments to cut scores, appears to be a reasonable way to assess symptom validity compared to other PVTs. Sensitivity and specificity of WMT IR scores may have been adversely impacted by lower sensitivity of other PVTs to independently identify invalid performance.

Performance validity (PVT) and symptom validity tests (SVT) have become standard practice in assessing credibility of neuropsychological profiles and symptom report. While PVTs assess cognitive task engagement, SVTs assess credibility of patient symptom report. Although prior research aimed to conceptualize the relationship between the two validity measure types, it generally focused on SVTs from the Minnesota Multiphasic Personality Inventory (MMPI-2 &RF) and the Structured Inventory of Malingered Symptoms (SIMS; Ord et al., 2021, MMPI-2; Van Dyke et al., 2013). Further studies have demonstrated mixed results, with many studies concluding that symptom and performance validity are separate but related constructs. The current study aimed to assess the relationship between PVTs and SVTs utilizing symptom validity measures from the Personality Assessment Inventory (PAI) across three samples, including neurodevelopmental, psychiatric, and traumatic brain injury groups.

Participants included 634 individuals consecutively referred for neuropsychological assessment who completed the Test of Memory Malingering (TOMM) and the PAI (mean Age = 41.7, SD = 15.7; mean Education = 13.7, SD = 2.7; 53% female; 89% Caucasian). Participants were divided into three groups based on referral, including neurodevelopmental (mean Age = SD = 10.7; mean Education = 13.4, SD = 2.5; 39% female; 79% Caucasian), psychiatric (mean Age = 44.7, SD = 15.0; mean Education = 13.8, SD = 2.8; 58% female; 90% Caucasian), and traumatic brain injury samples (mean Age = SD = 15.5; mean Education = 13.3, SD = 2.3; 50% female; 91% Caucasian). Four structural equation models (latent variable models) were constructed. The first model was fit across the entire sample while the remaining three were fit for the aforementioned subsamples. TOMM trials modeled the performance validity latent variable while SVTs from the PAI modeled the symptom validity latent variable (Positive Impression Management and Defensiveness Index modeled underreporting; Negative Impression Management, Malingering Index, and Cognitive Bias Scale modeled overreporting).

In the full sample model overreporting significantly predicted performance validity (p < 0.001, r = -0.31), indicating higher symptom overreporting related to poorer performance validity while symptom underreporting did not significantly predict performance validity (p = 0.09, r = 0.08). In the neurodevelopmental model overreporting did not significantly predict performance validity (p = 0.44, r = 0.10). Further, symptom underreporting did not significantly predict performance validity (p = 0.40, r = 0.10). Similarly, for the TBI model, overreporting did not significantly predict performance validity (p = 0.82, r = -0.02) and symptom underreporting did not significantly predict performance validity (p = 0.50, r = -0.08). For the psychiatric sample symptom underreporting did not significantly predict performance validity (p = 0.06, r = 0.11); however, symptom overreporting significantly predicted performance validity (p < 0.001, r = - 0.39).

The current study expands on prior research comparing the relationship between SVTs and PVTs in neuropsychological evaluation utilizing SVTs from the PAI. Results of the present study suggest the relationship between the SVTs and PVTs varies by referral type and further supports using both PVTs and SVTs in neuropsychological assessment.

While much research has demonstrated a relationship between depression and cognitive deficits, most studies have neglected to include measurements of performance validity. The very small number of studies that have examined this relationship when accounting for performance validity have found that the relationship between depression and cognition is small or nonsignificant. The current study examined the relationship between depression (assessed through both clinical interview and self-report symptom measures) and multiple domains of cognition after accounting for noncredible performance on neuropsychological testing.

Participants were veterans referred for outpatient clinical evaluation. Among other tests that varied across patients, the neuropsychological battery included: California Verbal Learning Test - second edition (CVLT-II) total immediate recall across trials 1–5, short delay free recall, and long delay free recall; Trail Making Test; FAS and Animal Fluency; Rey–Osterrieth Complex Figure Test (ROCF) copy and 3-minute delay recall; and Wisconsin Card Sorting Test (WCST) categories completed, total errors, and percent perseverative errors. These tests represent domains that have previously been examined in relation to depression (e.g., memory, processing speed, executive functioning). Evaluations were conducted for clinical purposes, so that some individuals who were not administered certain tests have missing data. The first set of regression analyses (N=206) included age, sex, and education at Step 1, Beck Depression Inventory-2 (BDI-2) total score at Step 2, and pass or failure of Trial 1 of the Test of Memory Malingering (TOMM) at Step 3 as predictors of performance on the 12 test indices. The second set of regression analyses (N=559) mirrored the first but with Major Depressive Disorder (MDD) diagnosis at Step 2 instead.

In the first set of analyses, after including TOMM in the model, only the relationship between BDI-2 and verbal fluency remained significant, but did not survive Bonferroni correction (p<.004). In the second set of analyses, before including the TOMM, MDD diagnosis was significantly related only to worse performance on Trails A and CVLT-II Short and Long Delay Free Recall, with small effect sizes (rp=.06–.15). When TOMM Trial 1 was included in the model, MDD diagnosis became a nonsignificant predictor of CVLT-II Long Delay Free Recall but remained a significant predictor for Trails A and CVLT-II Short Delay Free Recall (p<.05). After Bonferroni correction (p<.004), with TOMM Trial 1 included in the model, MDD diagnosis remained a significant predictor only of CVLT-II Short Delay Free Recall, with a small effect size (rp=.17).

After accounting for noncredible performance, there was little evidence for a relationship between depression diagnosis or symptoms and many cognitive domains. These results suggest that previously reported effects of depression on cognition are not mainly due to underlying neurological mechanisms, but rather to motivational factors. Future research could focus on the potential psychological mechanisms (e.g., negative attitudes, expectancy bias, low motivation, etc.) driving the relationship between depression and low effort on cognitive testing. If replicated, the current findings could be valuable to clinicians treating depressed individuals who have concerns about their cognitive functioning, by indicating psychoeducation and reassurance.

Although performance validity is critical in determining the quality and accuracy of test data, research suggests not all neuropsychologists incorporate performance validity tests (PVTs) in dementia evaluations (McGuire et al., 2019). Furthermore, well-validated embedded measures, such as Reliable Digit Span (RDS) from the Wechsler Adult Intelligence Scale - Fourth Edition (WAIS-IV), have evidenced an unusually high number of failures in a dementia population when utilizing typical clinical cut-offs (Zenisek et al., 2016). The objective of this study is to explore performance on embedded PVTs among older adults who have a major neurocognitive disorder (MND), specifically among Alzheimer disease (AD) and non-AD patients.

Archival data from outpatient neuropsychological evaluations were analyzed. All participants were at least 65 years of age, diagnosed with a MND, and completed Digit Span from the WAIS-IV, Brief Visuospatial Memory Test- Revised (BVMT-R), and Hopkins Verbal Learning Test-Revised (HVLT-R). In total, 84 participants, aged 67-96 (M=78.44, SD=6.11) with 6-20 years of education (M=13.47, SD=3.30), were included. The sample predominantly identified as female (n=60) and White (n=61). More individuals were diagnosed with AD (n=50) than non-AD dementia (n=34). Common non-AD diagnoses included Vascular (n=44), Lewy bodies (n=8), and Parkinson’s (n=2) dementias. Fisher’s Exact Test of Independence was used to account for the smaller sample to determine if there was a nonrandom association between diagnosis (AD vs non-AD) and embedded PVT performance: RDS< 7, BVMT-R Hits<4, BVMT-R Recognition Discrimination (RD) < 4, and HVLT-R RD < 5 (Bailey et al., 2018).

The Fisher’s Exact Test of Independence revealed a statistically significant association between neurocognitive diagnosis and RDS (p= .008), BVMT-R RD (p<.001), and HVLT-R RD (p<.001). BVMT-R Hits were not significantly associated with diagnosis (p = 0.10). These measures evidenced opposite patterns with RDS demonstrating a higher percentage of fails for the non-AD (63%) versus AD (20%) group. The AD group had a higher percentage of fails for BVMT-R RD (58% for AD and 13% for non-AD groups) and HLVT-R RD (66% for AD and 29% for non-AD group).

The current study suggests performance on embedded PVTs vary across MND diagnoses. Individuals with a non-AD diagnosis were more likely to fail RDS than those with AD. This is likely secondary to attention and working memory demands that are mediated by the frontal-subcortical networks, which are less impacted by AD pathology (Bonelli & Cummings, 2022; Loring et al., 2016). In contrast, AD patients were more likely to fail embedded PVTs within memory measures, which are largely mediated by the mesial temporal cortex associated with AD (Pluta, 2022). These results suggest embedded measures operate differently based on diagnosis and neuroanatomical systems affected. The clinical relevance of these findings includes potentially using alternative PVTs or different cut-offs based on diagnosis. Future research should attempt to better delineate more appropriate, as well as time efficient, PVTs among the dementia population.

Adequate effort by examinees during neurocognitive testing is a prerequisite to valid interpretation of test results. Utilizing performance validity tests (PVTs) is strongly recommended within pediatric mild Traumatic Brain Injury (mTBI) populations. PVTs have historically been created based on majority-white and monolingual groups; investigating their validity in additional patient populations remains essential. The Automatized Sequencing Task (AST) was developed as a brief validity measure within mTBI youth (Kirkwood, et.al., 2014). This study aimed to examine the clinical utility of the AST among youth identifying as Hispanic/Latino and/or bilingual within a mTBI clinical sample.

Participants ages 8-17 (N=103, M age=14.08, SD=2.2, 51.5% male, 42.7% Hispanic/Latino, 23.6% bilingual) were drawn from an outpatient mTBI/concussion program within the past 2.6 years. Median time of evaluation since injury was 3.7 weeks. Eligibility criteria included: 1) evaluated for a mTBI (GCS . 13) and 2) 8 through 17 years of age. Language status included English only and English-Spanish bilingual youth. Of the bilingual youth, 95% were considered English dominant. Youth were timed while reciting four well-learned (i.e., automatized) sequences as rapidly as possible: 1) the alphabet, 2) counting from 1-20, 3) the days of the week, and 4) the months of the year. Pass rates for the AST were examined using chi-square tests to compare performance based on ethnic/cultural identity (Hispanic vs Non-Hispanic), language status, age (children 8-12; teens 13-17), and gender.

In the clinical sample, 11.7% (n=12) could not complete AST months; 75% of non-completers were Hispanic/Latino. Participants who identified as Hispanic/Latino compared to Non-Hispanic/Latino participants were significantly more likely to fail the 4-item AST, χ2(1) =4.3, p<.05. The odds of failing the 4-item AST was 2.3 times higher if patients identified as Hispanic/Latino. Further, patients identifying as bilingual were even more likely to fail the 4-item AST, χ2(1) =4.5, p<.05. The odds of failing the 4-item AST was 3.0 times higher if patients were bilingual. There were no ethnicity or bilingual group differences in AST failure when examining performance on the 3-item AST. Neither age nor gender were a significant predictor of failure on the 3-item or 4-item AST.

Results suggest that the month item on the AST does not function consistently across Hispanic/Latino and bilingual youth. It cannot be presumed to be ‘automatic’ as a significant number of Hispanic/Latino and/or bilingual patients were unable to complete the month item, but with otherwise intact performance on the first three items. Administering only the first three items on the AST appears to be a more culturally sensitive alternative given the increased odds of 4-item failure in Hispanic/Latino and bilingual youth. Additional research is needed to explore the predictive validity of the AST as a PVT in varying ethnic, culturally, linguistically, and socioeconomically diverse mTBI pediatric populations.

We developed the Shell Game Task (SGT) as a novel Performance Validity Test (PVT). While most PVTs use a forced-choice paradigm with “memory” as the primary domain being assessed, the SGT is a face-valid measure of attention and working memory. We explored the accuracy of the SGT to detect noncredible performance using a simulatordesign study.

Ninety-four university students were randomly assigned to either best effort (CON) (n=49) or simulating traumatic brain injury (TBI) (SIM) (n=45) conditions. Participants completed a full battery of neuropsychological tests to simulate an actual evaluation, including the Test of Memory Malingering (TOMM) and the SGT. The SGT involves three cups and a red ball shown on the screen. Participants watch as the ball is placed under one of the three cups. Cups are then shuffled. Participants are asked to track the cup that contains the ball and correctly identify its location. We created two difficulty levels (easy vs hard, 20 trials each) by changing the number of times the cups were shuffled. Participants were given feedback (correct vs incorrect) after each trial. At the conclusion of the study, participants were asked about adherence to study directions they were given.

Participants with missing data (CON=1; SIM=2) or who reported non-adherence to study directions (CON=2; SIM=1) were removed from analyses. Twenty-five percent in SIM and 0% in CON failed TOMM

Trial 2 (<45) suggesting adequate manipulation of groups. Groups were not different in age, gender, ethnicity, or education (all p’s>.05). There were 9 participants in each group with concussion/TBI history. TBI history was not significantly related to performance on the SGT in either group, although participants with TBI history tended to do better. Average performances on TOMM Trial 1 (36.62 vs 47.91, p<.001) and TOMM Trial 2 (37.50 vs 49.71, p<.001) were significantly lower in the SIM group. Performance on SGT was also significantly lower in the SIM group across SGT Total Correct (20.17 vs 24.65 of 40, p=.008), SGT Easy (10.60 vs 13.52 of 20, p=.002), and SGT Hard (9.57 vs 11.13 of 20, p=.068). Mixed ANOVA showed a trend towards significant group by SGT difficulty interaction (F(1,86)=3.41, p=.052, np2=.043). There was steeper decline in performance on SGT Hard compared to SGT Easy for CON. ROC analyses suggested adequate but not ideal sensitivity/specificity: scores <8 on SGT Easy (sensitivity=26%; false positive=11%), <7 on SGT Hard (sensitivity=26%; false positive=7%), and <15 on SGT Total (sensitivity=24%; false positive=9%).

These preliminary data indicate the SGT may be able to detect malingered TBI. However, additional development of this measure is necessary. Further refinement of difficulty level may improve sensitivity/specificity (e.g., CON mean performance for SGT Easy trails was 13.52, suggesting the items may be too difficult). This study was limited to an online administration due to COVID, which could have affected results; future studies should test inperson administration of the SGT. In addition, performance in clinical control groups (larger samples of individuals with mild TBI, ADHD) should be tested to better determine specificity for these preliminary cutoffs.

This retrospective study compared base rates of failure on a series of standalone and embedded performance validity tests (PVTs) in a sample of Spanish-speaking forensic litigants and explored the impact of demographic factors on PVT performance.

62 Spanish-speaking participants involved in litigation (primarily for work-related mTBI) underwent outpatient neuropsychological evaluation. Country of origin spanned South American (56.5%), Caribbean (22.5%), Central American (16.1%), North American (3.2%), and Spanish (1.6%) regions. Of this sample, 56 completed the Test of Memory Malingering (TOMM), 45 completed the Rey Fifteen Item Test (RFIT), and 49 completed the Dot Counting Test (DCT). Embedded validity measures, Reliable Digit Span (RDS) and the WHO-Auditory Verbal Learning Test (WHO-AVLT) were completed by 32 and 48 participants, respectively.

Effects of age (M=42.4, SD=11.72) and region of origin did not significantly impact overall performance on any measure. Mean scores across all standalone PVTs were below cutoffs, which have been previously suggested for use with Spanish-speaking populations (TOMM <40, RFIT total <21, DCT e-score >18). Overall base rates of failure were as follows: 52.5% TOMM (T1 M= 37.5, SD=10.7; T2 M=35.1, SD=10.6), 64.9% RFIT (M=17.8, SD=7.8) 57.6% DCT (M=18.3, SD=8.8), 51.1% RDS (M=6.1, SD=1.6), 29.4% WHO-AVLT (M=10.7, SD=3.9. Years of education (M=9.98, 3.96) was significantly correlated with RFIT total score (r(43) = .48, p<.01) and DCT e-score (r(47) = -.34, p<.05. When stratified by level of education (0-6, 7-11, and 12+), a large discrepancy in base rate of failure was observed on the RFIT, with failures in 92% of participants with less than six years of education, as compared to 52% and 59% failure in those with 7-11 and 12+ years, respectively. Variability in base rates of DCT failure across levels of education, although less extreme than on the RFIT, again demonstrated higher rates of failure in participants with less than six years of education (0-6: 71%, 7-11: 54%, 12+: 52%).

These findings add to the existing literature surrounding measurement of suboptimal effort in Spanish-speaking populations. Base rates of PVT failure on both standalone and embedded measures were generally much higher than those reported in prior studies of forensic or compensation-seeking groups, including some with Spanish speaking participants. These high rates of failure are likely attributable, at least in part, to sample characteristics, due to the high proportion of individuals engaged in litigation associated with workplace injuries on construction sites at the study location. Such findings illustrate the importance of a thorough effort assessment for this population. Finally, results demonstrating reduced specificity of the RFIT in Spanish-speaking participants with less than six years of education, suggesting caution is warranted for its use in neuropsychological evaluations with such individuals.

Although studies have shown unique variance contributions from performance invalidity, it is difficult to interpret the meaning of cognitive data in the setting of failed performance validity tests (PVT). Furthermore, a clearer understanding of the clinical utility of cognitive data in the context of invalid PVTs is necessary to inform decisions about battery length once PVTs are failed. The primary aim of the current study is to broadly describe cognitive outcomes in the setting of PVT failure.

Two hundred and twenty-two veterans with a history of mild traumatic brain injury referred for clinical evaluation completed cognitive and performance validity measures. Standardized scores were characterized as Within Normal Limits and Below Normal Limits at the normative 16th percentile and number of Within Normal Limits scores were calculated for each participant. Cognitive outcomes are described across four commonly used PVTs. Rates of below normal limits cognitive performance, and PVT failure were assessed via student’s t tests among participants who were classified as productive or unproductive based on involvement in work and/or school.

Among participants who performed in the invalid range on TOMM trial 1, 36-81% of cognitive data reflected within normal limits performance. Similarly, 47-81% of those who demonstrated performance invalidity based on the Word Memory Test (WMT) earned broadly within normal limits scores across cognitive testing. For those with invalid performance based on the normative digit span scaled score, 35-88% of cognitive data was at or above the 16th percentile. Within normal limits across cognitive tests ranged from 16-71% when the California Verbal Learning Test-Second Edition forced choice was used as an indicator of performance validity. In the context of PVT failure, the average number of cognitive performances below the 16th percentile ranged from 5-7 of 14 tasks depending on which PVT measure was applied. Within the total sample, there were no differences in the total number of below normal limits performances on cognitive measures between productive and unproductive participants (T = 1.65, p = 1.00). Additionally, there were no differences in the total number of PVTs failed between the productive and unproductive groups (T = 0.33, p = 0.743).

Results of the current study suggest that the range of within normal limits cognitive performance in the context of failed performance validity measures varies greatly. Importantly, findings indicate that neurocognitive data may still provide important practical information regarding cognitive abilities (i.e., that test takers can oftentimes perform within broadly normal limits on many cognitive tasks), despite poor PVT outcomes. Further, given that neither rates of below normal limits cognitive performance nor rates of PVT failures differed among productivity groups, results have important implications for decisions to continue testing and recommendations in a clinical setting.

Some RCFT indices are effective Performance Validity Test (PVTs) during neuropsychological evaluations. A combination score that includes the copy score, true positive recognition, and atypical errors has proven to be especially useful (see Lu et al, 2003). However, this score was derived from administration that deviated from protocols outlined by Meyers & Meyers (1995) in that the Recognition trial was administered after the 3-minute delay instead of the 30-minute delay. The current study examined the utility of the RCFT combination score as a performance validity test (PVT) when completing the recognition trial after the 30-minute delay.

This study utilized archival data from 298 Veterans who presented for a clinical neuropsychological evaluation at a southern Veterans Affairs Medical Center. The evaluation included up to nine PVTs and all trials of the RCFT (per Meyers & Meyers, 1995). Patients were considered credible if all PVT performance fell within normal limits. This resulted in 232 patients in the credible group (Mage = 52.9 years, SDage = 15.2, Medu = SDedu = 2.5, 88% male, 71.2% White, 28.3% Black/African American). Patients were considered non-credible if they failed >2 PVTs. This resulted in 66 patients in the non-credible group (Mage = 51.6, SDage = 13.79, Medu = SDedu = 2.4, 92.4% male, 56.1% White, 43.9% Black/African American). Group assignment was also clinically confirmed. Receiver operating characteristic (ROC) curve analyses were conducted to discriminate between credible and non-credible groups utilizing the established RCFT combination score.

RCFT combination scores distinguished groups, with credible participants scoring higher than non-credible participants (F[1, 296]=63.76, p<.001, d=1.11; M = 56.9, SD = 9.3 vs. M = 46.5, SD = 9.5, respectively). A ROC analysis indicated AUC = .800 (95% CI = .73 to .86). When specificity was set at >90%, a cut-score of <46.5 yielded sensitivity at 46.0%. The analogous cut-score from the Lu et al. (2003) study (i.e., <47) was associated with a specificity of 88.7 and sensitivity of 46.0% in the current study.

As the Lu et al. (2003) established the combination score of the RCFT with procedures that deviated from the standardized protocol outlined by Meyers and Meyers (1995), clinicians who opted to adhere to Meyers and Meyers’ full protocol may have concerns about using the combination score as a PVT. The current study established a similar cut-off score to what Lu et al., (2003) reported (i.e., <46.5 vs. <47) while following a different administration procedure of the RCFT. Also, the index was moderately sensitive in the current study (i.e., 45.5%) but less so than what Lu et al. reported when using a cut-score that had >90% specificity (i.e., 75.9% sensitivity). This suggests that the index may be robust to deviations in administration procedures. Difference in sensitivity could be related to difference between samples. As the current sample was derived from a clinical, VA setting, current findings extend the generalizability of the index. Future research would benefit exploring if any subgroups would benefit from adjusted cut-scores to reduce the risk of false positive identification.

Empirical support for inclusion of performance validity testing (PVTs) in neuropsychological assessment continues to grow (Sweet et al., 2021). However, considerable validation is still needed to understand the impact of culturally mediated factors on the reliability of current, commonly used PVTs to accurately classify effort among various cultural groups. This study sought to contribute to the literature by examining the utility of several PVTs in a non-clinical, community-dwelling sample in Kampala, Uganda.

Participants included 52 residents (25 Female, 27 Male) who were born between 1953-2003 from the Wabigalo community of central Kampala. Individuals were recruited by community leaders and volunteered to participate. All 52 participants were administered the Dot Counting Test (DCT; Boone et al., 2002), Test of Memory Malingering (TOMM; Tombaugh, 1997), and Rey 15-Item Memorization Test (Rey 15; Lezak, 1995). Twenty-five participants also completed Green’s Non-Verbal Medical Symptom Validity Test (NV-MSVT; Green, 2006). Data from three participants was excluded due to suspected memory concerns. Instructions for all tests were translated into Luganda by a professional translator with experience in Luganda and were administered by Luganda-speaking individuals.

Using test manual-derived cut scores, 71.4% (n = 35) participants scored in the invalid range on the DCT, 10.2% (n = 5) produced total combined scores in the invalid range on Rey 15, 6.1% (n = 3) failed TOMM Trial 2, and one participant (4.3%) exceeded cut-offs on Green’s NV-MSVT.

In this non-clinical sample, manual cutoffs for DCT contributed to a high type-1 error rate. These findings suggest that culturally mediated factors may contribute to differences in engagement or performance on DCT. Future studies should explore these factors and continue to examine the utility of widely used tests in diverse samples.

The Test of Memory Malingering (TOMM) is a performance validity test (PVT) that aims to assess whether participants are giving adequate effort to perform well on tasks of memory performance (Tombaugh, 1996). Other PVTs, specifically the Forced Choice Recognition Trial in the California Verbal Learning Test, have shown that even single errors may indicate invalid performance (Erdodi et al., 2018). Finally, youth are often understudied in the PVT literature, and athletes are at increased risk of invalid performance on baseline testing due to many wanting to return to play following concussion (Erdal, 2012). Therefore, the objective of the current study is to examine whether single errors on TOMM Trial 1 are indicative of lower, and possibly invalid, cognitive performance in a youth sample, given that cognitive performance declines with even small decreases in effort (Green, 2007).

Healthy youth athletes (n=174) aged 8-16 years (M=12.07) completed the TOMM as well as other neuropsychological measures during baseline neuropsychological evaluation in a clinical research program for sports concussion. Independent samples t-tests compared youth athletes who scored 49 points on the TOMM (n=28) to youth athletes who scored a perfect 50 (n=50) across several groupings of neuropsychological measures. Participants who scored less than 49 or who didn’t complete the TOMM were excluded from the analyses.

Participants scoring 50/50 on TOMM Trial 1 scored significantly higher on Stroop Color Naming task (p=0.036), Verbal Learning Delayed task from the second edition of the Wide Range Assessment of Learning and Memory (WRAML-2, p=0.018), and Letter Number Sequencing task from the Weschler Intelligence Scale for Children (WISC-IV, p=0.025), relative to participants scoring 49/50. Though not statistically significant, results also showed a trend toward participants scoring 50/50 scoring higher on nearly every test in the battery.

Participants with a single error on TOMM Trial 1, as compared to participants with a perfect score, performed significantly worse on a processing speed task, a verbal learning task, and a working memory task as part of a comprehensive neuropsychological battery. The single-error group also trended toward scoring lower on nearly all of the remaining attention, processing speed, perceptual ability, memory, and executive functioning tasks in the battery. The results could lead to a more liberal interpretation of TOMM scores, given that the trend towards lower performance may be due to putting forth significantly less effort. These results point to the need for a similar comparison of the TOMM in a larger sample size, as greater power may reveal even more significant differences in performance. Findings also emphasize the importance of viewing performance validity on a continuum rather than as a dichotomous pass/fail. Understanding the TOMM and how single errors may be indicative of poorer performance in a youth sample could help to reframe the way PVT results are interpreted in clinical and forensic settings.

Assessment of response validity is essential to neuropsychological assessment. Although informant report of examinee functioning has previously been associated with examinee-generated performance and cognitive symptom invalidity (PVT; SVT-C), empirically-derived guidelines for interpreting informantreport validity are lacking. This study sought to assess the classification accuracy of a widely used informant-report measure, the Dementia Severity Rating Scale (DSRS), for discriminating examinee-generated PVT and SVT-C.

Data were collected from 145 examinee-informant dyads who completed neuropsychological batteries as part of a routine workup in an epilepsy monitoring unit. PVT status was determined by below-threshold performances on >2 indicators (Test of Memory Malingering, Wechsler Digit Span Age Corrected Scaled Score, Word Memory Test). SVT-C status was determined by above-threshold responses on both the Minnesota Multiphasic Personality Inventory-2-Restructured Form Response Bias Scale (MMPI-2-RF RBS) and Structured Inventory of Malingered Symptomatology Amnestic Disorders subscale (SIMS-AM). After assessing demographic and relational covariance via t-test and chi square analyses, receiver operator characteristic curves were derived to assess the classification accuracy of the DSRS for discriminating examinee PVT and SVT-C status.

DSRS total score demonstrated acceptable accuracy in classifying PVT status (AUC = .77), with cut scores of >21 and >15 yielding .93-.82 specificity and .44-.63 sensitivity. The DSRS also classified SVT-C status with acceptable accuracy (AUC = .71), with the aforementioned cut scores exhibiting .90-.78 specificity and .50-.64 sensitivity. The DSRS also classified SVT-C status using only one indicator (i.e., MMPI-2-RF RBS or SIMS-AM) with acceptable accuracy (AUC = .71-.72), with the aforementioned cut scores exhibiting .92 specificity and .37-.42 sensitivity.

The DSRS can be used to classify examinee-generated PVT and SVT-C on an epilepsy monitoring unit. Results provide empirically-derived psychometric guidelines for interpreting informant-report response validity that are clinically useful and lay the groundwork for future investigations of informant-report response validity.

Individuals with chronic pain frequently report diminished cognitive functioning. Prior cross-sectional studies have demonstrated strong associations between chronic pain and neurocognitive impairment, most notably in memory, attention, processing speed, and executive functioning. However, there is a paucity of research evaluating visual learning and memory abilities in this population. Further, while current practice standards advocate for the use of performance validity tests (PVTs) to assess the credibility of neuropsychological test performance, they have infrequently been incorporated into studies examining chronic pain samples, despite a higher observed rate of noncredible performance in the literature. This study aimed to compare visual learning and memory performance between a mixed neuropsychiatric (MNP) group and a chronic pain group in a validity-controlled sample.

The study consisted of 371 adults referred for outpatient neuropsychological evaluation. Between groups, various PVTs were administered, which included, at minimum, one freestanding and four embedded PVTs. All patients were administered the Brief Visuospatial Memory Test-Revised (BVMT-R) as part of a comprehensive neuropsychological evaluation. Only patients classified as valid performers (<1 PVT fails; n=295) were included in the analyses (Pain: n=109; MNP: n=186). The overall sample was 69% female and racially diverse (22% non-Hispanic Black, 43% non-Hispanic White, 30% Hispanic, 3% Asian/Pacific Islander, and 2% other race/ethnicities), with a mean age of 46.8 (SD=14.8) and mean education of 13.7 years (SD=2.7). Independent samples t-tests were performed to investigate the differences in visual learning and memory abilities between the chronic pain and MNP groups.

Chi-square analyses revealed significant differences between the pain and MNP groups on race, with more non-Hispanic White and Hispanic patients represented in the MNP group. There were also modest group differences in age and education. For the chronic pain group, patients scored lower on both BVMT-R Total T-Score (mean difference = 9.65T, p<.001) and BVMT Delayed Recall T-Score (mean difference = 8.97T, p<.001). The effect size was robust for both for Total T-Score (d = 0.682) and Delayed Recall T-Score (d = 0.632). In contrast, the difference in BVMT Recognition Discriminability was not statistically significant.

This study demonstrated significant differences in performance between mixed neuropsychiatric and chronic pain patients. Preliminary evidence indicated that chronic pain patients displayed lower visual mediated encoding and retrieval performance, although their recognition is comparable. Although the nature of this study was targeted toward visual learning and retrieval, it is likely that the known impact of chronic pain on attention, working memory, and processing speed accounts for this relationship. Future studies will benefit from further elucidating these potential mechanisms and better inform clinical decision-making and neuropsychological testing performance in patients with chronic pain.

As neuropsychologists aim to collect valid data, maximize the utility of assessments, make effective use of time, and best serve patient populations, measurement of performance validity is considered a critical issue for the field. As effort may vary across an evaluation, including performance validity tests (PVTs) throughout the assessment is important. Incorporating embedded PVTs in addition to free standing PVTs can be particularly useful in this regard. COWAT and animal naming are commonly administered verbal fluency measures. While there have been past investigations into their potential for detecting invalid performance, they are limited, and more research is needed. Perhaps most promising, Sugarman and Axelrod (2015) described a logistic regression derived formula utilizing the combined raw scores of COWAT and animal naming. The current study aimed to investigate the use of embedded PVTs within COWAT and animal naming to provide further support for the use of embedded PVTs in these measures.

All subjects were from a mixed clinical sample comprising military veterans from two VA Medical Centers in the northeast U.S., who were referred for neuropsychological evaluation. Subjects deemed credible had zero PVT failures. Subjects were considered non-credible performers if they failed at least two out of a possible eight PVTs administered. Subjects who failed one PVT were excluded from the study (n = 53). The final sample consisted of 116 individuals with credible performance (Mean Age = 35.5, SD = 8.8; Mean Edu = 13.6, SD = 2; Mean Est. IQ = 106, SD = 7.9) and 94 individuals with psychometrically determined non-credible performance (Mean Age = 38.5, SD = 9.4; Mean Edu = 113, SD = 2.1; Mean Est. IQ = 101, SD = 8.7). Performance of COWAT and animals in detecting non-credible performances was evaluated through calculation of classification accuracy statistics and use of the logistic regression formulas reported in Sugarman and Axelrod (2015).

For COWAT, the optimal cutoff was a raw score of <27 (specificity = 89%; sensitivity = 31%), and a T-score of <35 (specificity = 92%; sensitivity = 31%). For animal naming, optimal cutoffs were <16 for raw score (specificity = 92%, sensitivity = 38%) and <37 for T-score (specificity = 91%; sensitivity = 33%). The logistic regression formula based on raw scores for both COWAT and animal naming was inadequately sensitive at the recommended cutoff in this sample, but a coefficient of > .28 was revealed to be optimal (91% specificity; 42% sensitivity). When the formula for T-scores was used, a coefficient of > .38 was optimal (91% specificity; 28% sensitivity).

Results of the current research suggest that PVTs embedded within the commonly administered COWAT and animal naming verbal fluency tests can effectively detect low effort, in concordance with generally accepted standards. A logistic regression formula using raw scores in particular appears to be most effective, consistent with findings reported by Sugarman and Axelrod (2015).

Recent consensus guidelines have advocated for the use of multivariate performance validity assessment on ability-based measures such those used in neuropsychological assessment. Further, previous research has demonstrated that aggregating performance validity indicators may produce superior classification accuracy. The present study builds upon this research by aggregating data from three of the most commonly used performance validity measures (Test of Memory Malingering [TOMM], Rey Fifteen Item Test with recognition trial [FIT plus recognition], and Reliable Digit Span [RDS]) to create a performance validity composite measure in a veteran mild traumatic brain injury (mTBI) population.

Data of patients evaluated at a VA hospital who had completed the RDS, FIT plus recognition, and TOMM as part of their clinical neuropsychological evaluation were analyzed (n = 20). Two composite performance validity indexes were created: a Single Cutoff Performance Validity Index (SC-PVI), which measures the quantity of failures across performance validity measures (PVMs) by summing the total number of PVM failures, and a Multiple Cutoff Performance Validity Index (MC-PVI) which measures the number of failures as well as degree of failure(s) across measures of performance validity (e.g., a participant would attain a score of 3 if their PVM performance failed to reach a conservative cut point; they would obtain a score of 1 if they met conservative cut point, yet failed to reach a liberal cut point).

Only one participant (5%) attained a score of 0 on the SC-PVI (i.e., passing all PVTs using standard cutoffs) and MC-PVI (i.e., passing the most liberal cut points on all three PVMs). Conversely, eight participants (40%) attained a score of 3 on the SC-PVI (i.e., failed all three PVMs) and four participants (20%) attained a score of 9 (i.e., failed the most conservative cut points on all three PVMs). Results showed a significant (p < .001) ordinal association between the two indices (G = .984); however, there was no significant agreement between SC-PVI and MC-PVI models (k = -.087; p = .127).

Data revealed discordant findings between the three PVMs utilized. The majority of participants (75%) scored between 2-8 on the MC-PVI, meaning that they did not exceed all liberal cut points or fail all conservative cut points. These “grey area” scores suggest an indeterminate range of performance validity, which cannot be captured by a solitary cut point or neatly classified as pass or fail. The utility of multiple cutoff performance validity models (i.e., aggregating PVMs to consider the severity of failure and number of failures) is that they capture the nuance of these data when determining and discussing the credibility of a profile. Multiple cut point data also highlight how the choice of cutoff influences the outcome of performance validity research and clinical decision making. As such, future research on the classification accuracy of this MC-PVI is needed.

Traditional methods of assessing performance validity have numerous weaknesses, among them, results can be consciously manipulated by examinees who wish to feign cognitive impairment. This study tested the ability of pupillary dilation patterns during a performance validity test (PVT) to enhance diagnostic accuracy in discriminating true from feigned impairment of traumatic brain injury (TBI). Pupillometry provides information about physiological and psychological processes related to cognitive load, familiarity, and deception and is outside of conscious control. Patrick, Rapport, Kanser, Hanks, and Bashem (2021) established proof of concept for the utility of pupillometry with PVTs applied to the Test of Memory Malingering (TOMM). This study replicated and extended this work by evaluating the incremental utility of pupillary-derived indices on the Warrington Recognition Memory Test for Words (RMT).

Participants included 214 adults in three groups: adults with bona fide TBI (TBI; n = 51) healthy comparisons instructed to perform their best (HC; n = 72), and healthy adults instructed and incentivized to simulate cognitive impairment due to TBI (SIM; n = 91). Moreover, this study examined pupillary pattern differences among successful (i.e., failed < 1 PVT and performed impaired on cognitive tests) and unsuccessful (i.e., failed > 2 PVTs or did not score impaired on a cognitive test) SIM, including SIM who did and did not fail the RMT. The RMT was administered in the context of a comprehensive neuropsychological battery. Indices included two pure pupil dilation (PD) indices: a simple measure of baseline arousal (PD-Baseline) and a nuanced measure of dynamic engagement (PD-Range). A pupillo-behavioral index was also evaluated: Dilation-response inconsistency (DRI) captured the frequency with which examinees displayed a pupillary familiarity response to the correct answer but selected the unfamiliar stimulus (incorrect answer).

The results generally replicated Patrick et al. (2021), as all three indices were useful in discriminating between groups and provided incremental utility to traditional accuracy scores. PD-Baseline appeared sensitive to oculomotor dysfunction due to TBI (i.e., increasing accurate identification of that group); adults with TBI displayed significantly lower chronic arousal as compared to the two groups of healthy adults (SIM, HC). In fact, the TBI group showed significantly lower PD-Baseline than both unsuccessful simulators who were detected as feigners and successful simulators who passed PVTs but effectively feigned TBI on other tests. Dynamic engagement (PD-Range) yielded a hierarchical structure such that SIM were more dynamically engaged than TBI followed by HC. As predicted, simulators engaged in DRI significantly more frequently than other groups. Moreover, DRI added unique information to RMT accuracy in classifying unsuccessful simulators from all other groups. Each of these three pupillary indices showed large effect sizes, and logistic regressions indicated that each contributed unique variance in predicting group membership on one or more of the paired contrasts (i.e., SIM-TBI, SIM-HC, HC-TBI).

Taken together, the findings support continued research on the application of pupillometry to performance validity assessment: Pupillometry provided unique information in enhancing classification accuracy beyond traditional PVT accuracy scores. Overall, the findings highlight the promise of biometric indices in multimethod assessments of performance validity.

Few to no studies have directly compared the relative classification accuracies of the memory-based (Brief Visuospatial Memory Test-Revised Recognition Discrimination [BVMT-R RD] and Rey Auditory Verbal Learning Test Forced Choice [RAVLT FC]) and non-memory based (Reliable Digit Span [RDS] and Stroop Color and Word Test Word Reading trial [SCWT WR]) embedded performance validity tests (PVTs). This study’s main objective was to evaluate their relative classification accuracies head-to-head, as well as examine how their psychometric properties may vary among subgroups with and without genuine memory impairment.

This cross-sectional study included 293 adult patients who were administered the BVMT-R, WAIS-IV Digit Span, RAVLT and SCWT during outpatient neuropsychological evaluation at a Midwestern academic medical center. The overall sample was 58.0% female, 36.2% non-Hispanic White, 41.3% non-Hispanic Black, 15.7% Hispanic, 4.8% Asian/Pacific Islander, and 2.0% other, with a mean age of 45.7 (SD=15.8) and a mean education of 13.9 years (SD=2.8). Three patients had missing data, resulting in a final sample size of 290. Two hundred thirty-three patients (80%) were classified as having valid neurocognitive performance and 57 (20%) as having invalid neurocognitive performance based on performance across four independent, criterion PVTs (i.e., Test of Malingering Memory Trial 1, Word Choice Test, Dot Counting Test, Medical Symptom Validity Test). Of those with valid neurocognitive performance, 76 (48%) patients were considered as having genuine memory impairment through a memory composite band score (T<37 for (RAVLT Delayed Recall T-score + BVMT-R Delay Recall T-score/2).

The average memory composite band score for valid neurocognitive scores was T = 49.63 as compared to T = 27.57 for genuine memory impairment individuals. Receiver operating characteristic [ROC] curve analyses yielded significant areas under the curve (AUCs=.79-.87) for all four validity indices (p’s < .001). When maintaining acceptable specificity (91%-95%), all validity indices demonstrated acceptable yet varied sensitivities (35%-65%). Among the subgroup with genuine memory impairment, ROC curve analyses yielded significantly lower AUCs (.64-.69) for three validity indices (p’s < .001), except RDS (AUC=.644). At acceptable specificity (88%-93%), they yielded significantly lower sensitivities across indices (19%-39%). In the current sample, RAVLT FC and BVMT-R RD had the largest changes in sensitivities, with 19% and 26% sensitivity/90%-92% specificity at optimal cut-scores of <10 and <2, respectively, for individuals with memory impairment, compared to 65% and 61% sensitivity/94% specificity at optimal cut-scores of <13 and <4, respectively, for those without memory impairment.

Of the four validity scales, memory-based embedded PVTs yielded higher sensitivities while maintaining acceptable specificity compared to non-memory based embedded PVTs. However, they were also susceptible to the greatest declines in sensitivity among the subgroup with genuine memory impairment. As a result, careful consideration should be given to using memory-based embedded PVTs among individuals with clinically significant memory impairment based on other sources of information (e.g., clinical history, behavioral observation).