Hostname: page-component-78c5997874-fbnjt Total loading time: 0 Render date: 2024-11-19T02:52:28.113Z Has data issue: false hasContentIssue false
  • English
  • Français

Screening Tools for Cognitive Impairment in Adults with Substance Use Disorders: A Systematic Review

Published online by Cambridge University Press:  26 August 2021

Katherine Y. Ko ,
Nicole Ridley ,
Shayden D. Bryce Open the ORCID record for Shayden D. Bryce [Opens in a new window] ,
Kelly Allott ,
Angela Smith  and
Jody Kamminga Open the ORCID record for Jody Kamminga [Opens in a new window]
Katherine Y. Ko
Affiliation:
Department of Psychology, Macquarie University, Sydney, New South Wales, Australia
Nicole Ridley
Affiliation:
Drug and Alcohol Services, South Eastern Sydney Local Health District, Sydney, New South Wales, Australia
Shayden D. Bryce
Affiliation:
Centre for Youth Mental Health, The University of Melbourne, Parkville, Victoria, Australia Orygen, Parkville, Victoria, Australia
Kelly Allott
Affiliation:
Centre for Youth Mental Health, The University of Melbourne, Parkville, Victoria, Australia Orygen, Parkville, Victoria, Australia
Angela Smith
Affiliation:
Hunter New England Health Libraries, Hunter New England Local Health District, Newcastle, New South Wales, Australia
Jody Kamminga*
Affiliation:
Drug and Alcohol Clinical Services, Hunter New England Local Health District, Newcastle, New South Wales, Australia
*
*Correspondence and reprint requests to: Jody Kamminga, Drug and Alcohol Clinical Services, Hunter New England Local Health District, Level 3, 670 Hunter Street, Newcastle West, NSW 2304, Australia. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives:

Cognitive impairment is common in individuals with substance use disorders (SUDs), yet no evidence-based guidelines exist regarding the most appropriate screening measure for use in this population. This systematic review aimed to (1) describe different cognitive screening measures used in adults with SUDs, (2) identify substance use populations and contexts these tools are utilised in, (3) review diagnostic accuracy of these screening measures versus an accepted objective reference standard, and (4) evaluate methodology of included studies for risk of bias.

Methods:

Online databases (PsycINFO, MEDLINE, Embase, and CINAHL) were searched for relevant studies according to pre-determined criteria, and risk of bias and applicability was assessed using the Quality Assessment of Diagnostic Accuracy Studies–2 (QUADAS–2). At each review phase, dual screening, extraction, and quality ratings were performed.

Results:

Fourteen studies met inclusion, identifying 10 unique cognitive screening tools. The Montreal Cognitive Assessment (MoCA) was the most common, and two novel screening tools (Brief Evaluation of Alcohol-Related Neuropsychological Impairments [BEARNI] and Brief Executive Function Assessment Tool [BEAT]) were specifically developed for use within SUD populations. Twelve studies reported on classification accuracy and relevant psychometric parameters (e.g., sensitivity and specificity). While several tools yielded acceptable to outstanding classification accuracy, there was poor adherence to the Standards for Reporting Diagnostic Accuracy Studies (STARD) across all studies, with high or unclear risk of methodological bias.

Conclusions:

While some screening tools exhibit promise for use within SUD populations, further evaluation with stronger methodological design and reporting is required. Clinical recommendations and future directions for research are discussed.

Keywords

CognitionScreeningAdultsSubstance-related disordersNeuropsychology
Type
Critical Review
Information
Journal of the International Neuropsychological Society , Volume 28 , Issue 7 , August 2022 , pp. 756 - 779
Copyright
Copyright © INS. Published by Cambridge University Press, 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Abu-Akel, A., Bousman, C., Skafidas, E., & Pantelis, C. (2018). Mind the prevalence rate: overestimating the clinical utility of psychiatric diagnostic classifiers. Psychological Medicine, 48, 1225–1227. doi: 10.1017/S0033291718000673 CrossRefGoogle ScholarPubMed
Aharonovich, E., Amrhein, P.C., Bisaga, A., Nunes, E.V., & Hasin, D.S. (2008). Cognition, commitment language, and behavioral change among cocaine-dependent patients. Psychology of Addictive Behaviors: Journal of the Society of Psychologists in Addictive Behaviors, 22, 557–562. doi: 10.1037/a0012971 CrossRefGoogle ScholarPubMed
American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders (5th ed.). Washington, DC: Author.Google Scholar
American Psychological Association Practice Organization. (2014). Distinguishing between screening and assessment for mental and behavioral health problems: a statement from an American Psychological Association Practice Organization work group on screening and psychological assessment. Retrieved from: https://www.apaservices.org/practice/reimbursement/billing/assessment-screening Google Scholar
Bates, M.E., Bowden, S.C., & Barry, D. (2002). Neurocognitive impairment associated with alcohol use disorders: implications for treatment. Experimental and Clinical Psychopharmacology, 10, 193–212. doi: 10.1037//1064-1297.10.3.193 CrossRefGoogle ScholarPubMed
Bates, M.E., Buckman, J.F., & Nguyen, T.T. (2013). A role for cognitive rehabilitation in increasing the effectiveness of treatment for alcohol use disorders. Neuropsychology Review, 23, 27–47. doi: 10.1007/s11065-013-9228-3 CrossRefGoogle ScholarPubMed
Berry, J., Shores, E.A., Nardo, T., Sedwell, A., Lunn, J., Marceau, E., …Batchelor, J. (2021). The brief executive-function assessment tool: a new cognitive impairment screening tool for alcohol and other drug services. Applied Neuropsychology: Adult. doi: 10.1080/23279095.2021.1895791 CrossRefGoogle ScholarPubMed
Bossuyt, P.M., Reitsma, J.B., Bruns, D.E., Gatsonis, C.A., Glasziou, P.P., Irwig, L., … Cohen, J.F. (2015). STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ: British Medical Journal, 351, h5527. doi: 10.1136/bmj.h5527 CrossRefGoogle ScholarPubMed
Bossuyt, P.M., Reitsma, J.B., Bruns, D.E., Gatsonis, C.A., Glasziou, P.P., Irwig, L.M., … Lijmer, J.G. (2003). The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Annals of Internal Medicine, 138, W1–W12.CrossRefGoogle ScholarPubMed
Braatveit, K.J., Torsheim, T., & Hove, O. (2018). Screening for intellectual disabilities: a validation of the Hayes Ability Screening Index for in-patients with substance use disorder. Nordic Journal of Psychiatry, 72, 387–392. doi: 10.1080/08039488.2018.1468923 CrossRefGoogle ScholarPubMed
Bruijnen, C.J., Jansen, M., Dijkstra, B.A., Walvoort, S.J., Lugtmeijer, S., Markus, W., … & Kessels, R.P. (2019). The Montreal Cognitive Assessment (MoCA) as a cognitive screen in addiction health care: a validation study for clinical practice. Journal of Substance Use, 24, 47–54. doi: 10.1080/14659891.2018.1497102 CrossRefGoogle Scholar
Brodersen, C., Koen, E., Ponte, A., Sánchez, S., Segal, E., Chiapella, A., … & Lemberg, A. (2014). Cognitive function in patients with alcoholic and nonalcoholic chronic liver disease. The Journal of Neuropsychiatry and Clinical Neurosciences, 26, 241–248. doi: 10.1176/appi.neuropsych.12040091 CrossRefGoogle ScholarPubMed
Buckman, J.F., Bates, M.E., & Cisler, R.A. (2007). Social networks and their influence on drinking behaviors: differences related to cognitive impairment in clients receiving alcoholism treatment. Journal of Studies on Alcohol and Drugs, 68, 738–747. doi: 10.15288/jsad.2007.68.738 CrossRefGoogle ScholarPubMed
Buckman, J.F., Bates, M.E., & Morgenstern, J. (2008). Social support and cognitive impairment in clients receiving treatment for alcohol- and drug-use disorders: a replication study. Journal of Studies on Alcohol and Drugs, 69, 738–746. doi: 10.15288/jsad.2008.69.738 CrossRefGoogle ScholarPubMed
Bunnage, M. (2017). How do I know when a diagnostic test works? In Bowden, S. (Ed.), Neuropsychological assessment in the age of evidence-based practice: Diagnostic and treatment evaluation (pp. 223–238). New York, NY: Oxford University Press.Google Scholar
Carroll Chapman, S.L. & Wu, L.T. (2012). Substance abuse among individuals with intellectual disabilities. Research in Developmental Disabilities, 33, 1147–1156. doi: 10.1016/j.ridd.2012.02.009 CrossRefGoogle ScholarPubMed
Chen, L.Y., Strain, E.C., Crum, R.M., & Mojtabai, R. (2013). Gender differences in substance abuse treatment and barriers to care among persons with substance use disorders with and without comorbid major depression. Journal of Addiction Medicine, 7, 325–334. doi: 10.1097/ADM.0b013e31829b7afe CrossRefGoogle ScholarPubMed
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.Google Scholar
Cohen, J.F., Korevaar, D.A., Altman, D.G., Bruns, D.E., Gatsonis, C.A., Hooft, L., … Bossuyt, P.M. (2016). STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open, 6, e012799. doi: 10.1136/bmjopen-2016-012799 CrossRefGoogle ScholarPubMed
Copersino, M.L., Fals-Stewart, W., Fitzmaurice, G., Schretlen, D.J., Sokoloff, J., & Weiss, R.D. (2009). Rapid cognitive screening of patients with substance use disorders. Experimental and Clinical Psychopharmacology, 17, 337–344. doi: 10.1037/a0017260 CrossRefGoogle ScholarPubMed
Copersino, M.L., Schretlen, D.J., Fitzmaurice, G.M., Lukas, S.E., Faberman, J., Sokoloff, J., & Weiss, R.D. (2012). Effects of cognitive impairment on substance abuse treatment attendance: predictive validation of a brief cognitive screening measure. The American Journal of Drug and Alcohol Abuse, 38, 246–250. doi: 10.3109/00952990.2012.670866 CrossRefGoogle ScholarPubMed
Covidence Systematic Review Software. Veritas Health Innovation. Melbourne, Australia. Available at www.covidence.org Google Scholar
Cunha, P.J., Nicastri, S., de Andrade, A.G., & Bolla, K.I. (2010). The frontal assessment battery (FAB) reveals neurocognitive dysfunction in substance-dependent individuals in distinct executive domains: abstract reasoning, motor programming, and cognitive flexibility. Addictive Behaviors, 35, 875–881. doi: 10.1016/j.addbeh.2010.05.005 CrossRefGoogle ScholarPubMed
Daley, D.C. & Moss, H.B. (2009). Dual disorders: Counseling clients with Chemical Dependency and Mental Illness. Center City: Simon and Schuster.Google Scholar
Dubois, B., Slachevsky, A., Litvan, I., & Pillon, B. (2000). The FAB: a Frontal assessment battery at bedside. Neurology, 55, 1621–1626. doi: 10.1212/wnl.55.11.1621 CrossRefGoogle Scholar
Ewert, V., Pelletier, S., Alarcon, R., Nalpas, B., Donnadieu-Rigole, H., Trouillet, R., & Perney, P. (2018). Determination of MoCA cutoff score in patients with alcohol use disorders. Alcoholism: Clinical and Experimental Research, 42, 403–412 doi: 10.1111/acer.13547 CrossRefGoogle ScholarPubMed
Fals-Stewart, W. (1997). Detection of neuropsychological impairment among substance-abusing patients: accuracy of the Neurobehavioral Cognitive Status Examination. Experimental and Clinical Psychopharmacology, 5, 269.CrossRefGoogle ScholarPubMed
Folstein, M.F., Folstein, S.E., & McHugh, P.R. (1975). “Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189–198.CrossRefGoogle ScholarPubMed
Godefroy, O., Gibbons, L., Diouf, M., Nyenhuis, D., Roussel, M., Black, S., Bugnicourt, J.M., & GREFEX Study Group. (2014). Validation of an integrated method for determining cognitive ability: implications for routine assessments and clinical trials. Cortex, 54, 51–62. doi: 10.1016/j.cortex.2014.01.016 CrossRefGoogle ScholarPubMed
Gooden, J.R., Cox, C.A., Petersen, V., Curtis, A., Manning, V., & Lubman, D.I. (2020). Characterisation of presentations to a community-based specialist addiction neuropsychology service: cognitive profiles, diagnoses and comorbidities. Drug and Alcohol Review. doi: 10.1111/dar.13135 Google ScholarPubMed
Gould, T.J. (2010). Addiction and cognition. Addiction Science & Clinical Practice, 5, 4–14.Google ScholarPubMed
Grella, C.E., Hser, Y.I., Joshi, V., & Rounds-Bryant, J. (2001). Drug treatment outcomes for adolescents with comorbid mental and substance use disorders. The Journal of Nervous and Mental Disease, 189, 384–392.CrossRefGoogle ScholarPubMed
Grohman, K. & Fals-Stewart, W. (2004). The detection of cognitive impairment among substance-abusing patients: the accuracy of the Neuropsychological Assessment Battery–screening module. Experimental and Clinical Psychopharmacology, 12, 200. doi: 10.1037/1064-1297.12.3.200 CrossRefGoogle ScholarPubMed
Hagen, E., Erga, A.H., Hagen, K.P., Nesvåg, S.M., McKay, J.R., Lundervold, A.J., & Walderhaug, E. (2016). Assessment of executive function in patients with substance use disorder: a comparison of inventory- and performance-based assessment. Journal of Substance Abuse Treatment, 66, 1–8. doi: 10.1016/j.jsat.2016.02.010 CrossRefGoogle ScholarPubMed
Hayes, S.C. (2000). Hayes Ability Screening Index (HASI) Manual. Sydney: Faculty of Medicine, University of Sydney.Google Scholar
Horner, M.D., Harvey, R.T., & Denier, C.A. (1999). Self-report and objective measures of cognitive deficit in patients entering substance abuse treatment. Psychiatry Research, 86, 155–161.CrossRefGoogle ScholarPubMed
Katz, N., Itzkovich, M., Averbuch, S., & Elazar, B. (1989). Loewenstein occupational therapy cognitive assessment (LOTCA) battery for brain-injured patients: reliability and validity. American Journal of Occupational Therapy, 43, 184–192.CrossRefGoogle ScholarPubMed
Kelly, T.M. & Daley, D.C. (2013). Integrated treatment of substance use and psychiatric disorders. Social Work in Public Health, 28, 388–406. doi: 10.1080/19371918.2013.774673 CrossRefGoogle ScholarPubMed
Kiernan, R.J., Mueller, J., Langston, J.W., & Van Dyke, C. (1987). The Neurobehavioral Cognitive Status Examination: a brief but quantitative approach to cognitive assessment. Annals of Internal Medicine, 107(4), 481–485. doi: 10.7326/0003-4819-107-4-481 CrossRefGoogle ScholarPubMed
Levin, F.R. (2007). Diagnosing attention-deficit/hyperactivity disorder in patients with substance use disorders. Journal of Clinical Psychiatry, 68, 9–14.Google ScholarPubMed
Lin, E., Balogh, R., McGarry, C., Selick, A., Dobranowski, K., Wilton, A.S., & Lunsky, Y. (2016). Substance-related and addictive disorders among adults with intellectual and developmental disabilities (IDD): an Ontario population cohort study. BMJ Open, 6, e011638. doi: 10.1136/bmjopen-2016-011638 CrossRefGoogle ScholarPubMed
Mandrekar, J.N. (2010). Receiver operating characteristic curve in diagnostic test assessment. Journal of Thoracic Oncology, 5, 1315–1316. doi: 10.1097/jto.0b013e3181ec173d CrossRefGoogle ScholarPubMed
McGee, S. (2002). Simplifying likelihood ratios. Journal of General Internal Medicine, 17, 646–649. doi: 10.1046/j.1525-1497.2002.10750.x CrossRefGoogle ScholarPubMed
Mioshi, E., Dawson, K., Mitchell, J., Arnold, R., & Hodges, J.R. (2006). The Addenbrooke’s Cognitive Examination Revised (ACE–R): a brief cognitive test battery for dementia screening. International Journal of Geriatric Psychiatry, 21, 1078–1085. doi: 10.1002/gps.1610 CrossRefGoogle ScholarPubMed
Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., & The PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLOS Medicine, 6, e1000097. doi: 10.1371/journal.pmed1000097 CrossRefGoogle ScholarPubMed
Molinaro, A.M. (2015). Diagnostic tests: how to estimate the positive predictive value. Neuro-Oncology Practice, 2, 162–166. doi: 10.1093/nop/npv030 Google ScholarPubMed
Nasreddine, Z.S., Phillips, N.A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., … & Chertkow, H. (2005). The Montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53, 695–699. doi: 10.1111/j.1532-5415.2005.53221.x CrossRefGoogle ScholarPubMed
Owen, K.P., Sutter, M.E. & Albertson, T.E. (2014). Marijuana: respiratory tract effects. Clinical Reviews in Allergy & Immunology, 46, 65–81. doi: 10.1007/s12016-013-8374-y CrossRefGoogle ScholarPubMed
Paez, A. (2017). Gray literature: an important resource in systematic reviews. Journal of Evidence-Based Medicine, 10, 233–240. doi: 10.1111/jebm.12265 CrossRefGoogle ScholarPubMed
Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D. … & Shamseer, L. (2020). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. MetarXiv. https://osf.io/preprints/metaarxiv/v7gm2/ Google Scholar
Parry-Jones, B.L., Vaughan, F.L., & Miles Cox, W. (2006). Traumatic brain injury and substance misuse: a systematic review of prevalence and outcomes research (1994–2004). Neuropsychological Rehabilitation, 16, 537–560. doi: 10.1080/09602010500231875 CrossRefGoogle Scholar
Pelletier, S., Alarcon, R., Ewert, V., Forest, M., Nalpas, B., & Perney, P. (2018). Comparison of the MoCA and BEARNI tests for detection of cognitive impairment in in-patients with alcohol use disorders. Drug and Alcohol Dependence, 187, 249–253. doi: 10.1016/j.drugalcdep.2018.02.026 CrossRefGoogle ScholarPubMed
Ramey, T. & Regier, P. (2019). Cognitive impairment in substance use disorders. CNS Spectrums, 24, 102–113. doi: 10.1017/S1092852918001426 CrossRefGoogle ScholarPubMed
Ridley, N., Batchelor, J., Draper, B., Demirkol, A., Lintzeris, N., & Withall, A. (2018). Cognitive screening in substance users: diagnostic accuracies of the mini-mental state examination, Addenbrooke’s Cognitive Examination–Revised, and Montreal Cognitive Assessment. Journal of Clinical and Experimental Neuropsychology, 40, 107–122. doi: 10.1080/13803395.2017.1316970 CrossRefGoogle ScholarPubMed
Ritz, L., Lannuzel, C., Boudehent, C., Vabret, F., Bordas, N., Segobin, S., … & Beaunieux, H. (2015). Validation of a brief screening tool for alcohol-related neuropsychological impairments. Alcoholism: Clinical and Experimental Research, 39, 2249–2260. doi: 10.1111/acer.12888 Google ScholarPubMed
Roebuck-Spencer, T.M., Glen, T., Puente, A.E., Denney, R.L., Ruff, R.M., Hostetter, G., & Bianchini, K.J. (2017). Cognitive screening tests versus comprehensive neuropsychological test batteries: a National Academy of Neuropsychology Education Paper. Archives of Clinical Neuropsychology, 32, 491–498. doi: 10.1093/arclin/acx021 Google Scholar
Rojo-Mota, G., Pedrero-Pérez, E.J., Ruiz-Sánchez de León, J.M., León-Frade, I., Aldea-Poyo, P., Alonso-Rodríguez, M., … & Morales-Alonso, S. (2017). Loewenstein occupational therapy cognitive assessment to evaluate people with addictions. Occupational Therapy International, 1–7. doi: 10.1155/2017/2750328 CrossRefGoogle ScholarPubMed
Salavert, J., Clarabuch, A., Fernández-Gómez, M.J., Barrau, V., Giráldez, M.P., & Borràs, J. (2018). Substance use disorders in patients with intellectual disability admitted to psychiatric hospitalisation. Journal of Intellectual Disability Research: JIDR, 62, 923–930. doi: 10.1111/jir.12514 CrossRefGoogle ScholarPubMed
Schoenberg, M.R. & Rum, R.S. (2017). Towards reporting standards for neuropsychological study results: a proposal to minimize communication errors with standardized qualitative descriptors for normalized test scores. Clinical Neurology and Neurosurgery, 162, 72–79. doi: 10.1016/j.clineuro.2017.07.010 CrossRefGoogle ScholarPubMed
Schueler, S., Schuetz, G.M., & Dewey, M. (2012). The revised QUADAS–2 tool. Annals of Internal Medicine, 156, 323–324. doi: 10.7326/0003-4819-156-4-201202210-00018 CrossRefGoogle ScholarPubMed
Secades-Villa, R., García-Rodríguez, O., & Fernández-Hermida, J.R. (2015). Contingency management for substance use disorders in Spain: implications for research and practice. Preventive Medicine, 80, 82–88. doi: 10.1016/j.ypmed.2015.07.001 CrossRefGoogle ScholarPubMed
Slater, H. & Young, J. (2013). A review of brief cognitive assessment tests. Reviews in Clinical Gerontology, 23, 164–176. doi: 10.1017/s0959259813000038 CrossRefGoogle Scholar
Stavro, K., Pelletier, J., & Potvin, S. (2013). Widespread and sustained cognitive deficits in alcoholism: a meta-analysis. Addiction Biology, 18, 203–213. doi: 10.1111/j.1369-1600.2011.00418.x CrossRefGoogle ScholarPubMed
Stern, R.A., & White, T. (2003). NAB, Neuropsychological Assessment Battery: Administration, scoring, and interpretation manual. Lutz, FL: Psychological Assessment Resources.Google Scholar
To, W.T., Vanheule, S., Vanderplasschen, W., Audenaert, K., & Vandevelde, S. (2015). Screening for intellectual disability in persons with a substance abuse problem: exploring the validity of the Hayes Ability Screening Index in a Dutch-speaking sample. Research in Developmental Disabilities, 36, 498–504. doi: 10.1016/j.ridd.2014.10.046 CrossRefGoogle Scholar
Trevethan, R. (2017). Sensitivity, specificity, and predictive values: foundations, pliabilities, and pitfalls in research and practice. Frontiers in Public Health, 5. doi: 10.3389/fpubh.2017.00307 CrossRefGoogle ScholarPubMed
Verdejo-Garcia, A., Garcia-Fernandez, G., & Dom, G. (2019). Cognition and addiction. Dialogues in Clinical Neuroscience, 21, 281–290. doi: 10.31887/DCNS.2019.21.3/gdom CrossRefGoogle ScholarPubMed
Volkow, N.D., Baler, R.D., Compton, W.M., & Weiss, S.R. (2014). Adverse health effects of marijuana use. The New England Journal of Medicine, 370, 2219–2227. doi: 10.1056/NEJMra1402309 CrossRefGoogle ScholarPubMed
Wester, A.J., Westhoff, J., Kessels, R.P., & Egger, J.I. (2013). The Montreal Cognitive Assessment (MoCA) as a measure of severity of amnesia in patients with alcohol-related cognitive impairments and Korsakoff syndrome. Clinical Neuropsychiatry, 10, 134–141.Google Scholar
Whiting, P.F., Rutjes, A.W., Westwood, M.E., Mallett, S., Deeks, J.J., Reitsma, J.B., … QUADAS–2 Group. (2011). QUADAS–2: a revised tool for the quality assessment of diagnostic accuracy studies. Annals of Internal Medicine, 155, 529–536. doi: 10.7326/0003-4819-155-8-201110180-00009 CrossRefGoogle ScholarPubMed
Supplementary material: File

Ko et al. supplementary material

Table S1 and S2

Download Ko et al. supplementary material(File)
File 25.1 KB