The use of valid and practical screening scales might ease the burden for greatly needed universal testing for mental health, substance use and dual disorders, but do they work well with all populations? Do they miss correct identification of certain groups?

To understand discrepancies in diagnostic prediction using the AC-OK screen in conjunction with other standardized assessment scales.

Two hundred and twenty-six Latino participants were recruited from primary care and community clinics in Madrid, Barcelona and Boston and assessed with standardized mental health and substance abuse measures including the AC-OK screen and with a Computerized adaptive test for mental health (CAT-MH). A measure of frequency of discrepancies and an adjusted and unadjusted comparison of results and demographic characteristics or respondents were made for mental health, substance abuse or for discrepancies in both categories.

35.4% of cases were discrepant in mental health (AC-OK-Mental Health vs. Patient Health Questionnaire-9, Generalized Anxiety Disorder 7 or PTSD Checklist) and 14.2% in substance abuse (AC-OK-substance abuse vs. drug abuse screening test or Alcohol use disorders identification test). When CAT-MH scale was incorporated, discrepant results were found in 24.3% and 14.2%, respectively. No association was found between substance abuse discrepancies and patient demographics. In logit regressions being from Barcelona, of younger age and male were significant predictors of discrepancies.

Discrepancies were observed in the diagnostic prediction, with differences detected for site and sociodemographic characteristics of participants suggesting the importance of testing screeners for site and population differences. Evidence for the misclassification of young males is discussed. Caution is warranted in the implementation of screeners for at risk populations.

The authors have not supplied their declaration of competing interest.