Hostname: page-component-cd9895bd7-jn8rn Total loading time: 0 Render date: 2024-12-23T15:56:15.905Z Has data issue: false hasContentIssue false
  • English
  • Français

Diagnostic Accuracy of Transient Ischemic Attack from Physician Claims

Published online by Cambridge University Press:  07 February 2017

Jodi D. Edwards ,
Mieke Koehoorn ,
Lara A. Boyd ,
Boris Sobolev  and
Adrian R. Levy
Jodi D. Edwards*
Affiliation:
School of Population and Public Health, University of British Columbia, Vancouver, British Columbia, Canada
Mieke Koehoorn
Affiliation:
School of Population and Public Health, University of British Columbia, Vancouver, British Columbia, Canada
Lara A. Boyd
Affiliation:
Department of Physical Therapy, University of British Columbia, Vancouver, British Columbia, Canada
Boris Sobolev
Affiliation:
School of Population and Public Health, University of British Columbia, Vancouver, British Columbia, Canada
Adrian R. Levy
Affiliation:
School of Population and Public Health, University of British Columbia, Vancouver, British Columbia, Canada Department of Community Health and Epidemiology, Dalhousie University, Halifax, Nova Scotia, Canada
*
Correspondence to: Jodi Edwards, Heart and Stroke Foundation Postdoctoral Fellow, Canadian Partnership for Stroke Recovery, Sunnybrook Research Institute, M6, 2075 Bayview Avenue, Toronto, Ontario, Canada M4N 3M5. Email: [email protected].
Rights & Permissions [Opens in a new window]

Abstract

Background: Hospitalization data underestimate the occurrence of transient ischemic attack (TIA). As TIA is frequently diagnosed in primary care, methodologies for the accurate ascertainment of a TIA from physician claims data are required for surveillance and health systems planning in this population. The present study evaluated the diagnostic accuracy of multiple algorithms for TIA from a longitudinal population-based physician billing database. Methods: Population-based administrative data from the province of British Columbia were used to identify the base population (1992–2007; N=102,492). Using discharge records for hospital admissions for acute ischemic stroke with a recent (<90 days) TIA as the reference standard, we performed receiver-operating characteristic analyses to calculate sensitivity, specificity, positive and negative predictive values and overall accuracy, and to compare area under the curve for each physician billing algorithm. To evaluate the impact of different case definitions on population-based TIA burden, we also estimated the annual TIA occurrence associated with each algorithm. Results: Physician billing algorithms showed low to moderate sensitivity, with the algorithm for two consecutive physician visits within 90 days showing the highest sensitivity at 37.7% (CI95%=37.4–38.1). All algorithms demonstrated high specificity and moderate to high overall accuracy, resulting in low positive predictive values (≤5%), low discriminability (0.53–0.57) and high false positive rates (1 – specificity). Population-based estimates of TIA occurrence were comparable to prior studies and declined over time. Conclusions: Physician billing data have insufficient sensitivity to identify TIAs but may be used in combination with hospital discharge data to improve the accuracy of estimating the population-based occurrence of TIAs.

Résumé

Exactitude du diagnostic de l’ischémie cérébrale transitoire selon les déclarations des médecins.Contexte: Les données d’hospitalisation sous-estiment l’incidence de l’ischémie cérébrale transitoire (ICT). L’ICT est souvent diagnostiquée dans un contexte de soins de première ligne. Pour la surveillance et la planification des soins de santé dans cette population, il est donc nécessaire d’utiliser une méthodologie, qui soit basée sur les données des déclarations des médecins pour la détermination exacte du nombre de cas d’ICT. Cette étude a évalué l’exactitude du diagnostic de l’ICT au moyen d’algorithmes dans une base de données longitudinale de population pour la facturation des médecins. Méthodologie: Les données démographiques administratives de la Colombie-Britannique ont été utilisées pour définir la population à l’étude (1992-2007 ; n=102 492). Nous avons utilisé les données au moment du congé hospitalier des patients admis pour un accident vasculaire ischémique aigu en utilisant une ICT récente comme standard et référence. Nous avons procédé à des analyses de la fonction d’efficacité du récepteur pour calculer la sensibilité, la spécificité, les valeurs prédictives positives et négatives et l’exactitude globale ainsi que pour comparer la surface sous la courbe pour l’algorithme de facturation de chaque médecin. Résultats: Les algorithmes de facturation des médecins ont montré une sensibilité de faible à modérée, l’algorithme pour deux visites consécutives chez le médecin en dedans de 90 jours ayant la sensibilité la plus élevée, soit 37,7% (IC à 95% de 37,4 à 38,1). Tous les algorithmes avaient une spécificité élevée et une exactitude globale de modérée à élevée, avec des valeurs prédictives positives faibles (≤ 5%), un faible pouvoir discriminant (0,53 à 0,57) et des taux élevés de faux positifs (1 – la spécificité). Les estimés populationnels de l’incidence de l’ICT étaient comparables à ceux des études antérieures et diminuaient avec le temps. Conclusions: les données de facturation des médecins ont une sensibilité insuffisante pour identifier les ICT mais peuvent être utilisées conjointement avec les données du congé hospitalier pour améliorer l’exactitude de l’estimation de l’incidence populationnelle de l’ICT.

Keywords

Transient ischemic attackdiagnostic methodpopulation studiesepidemiology
Type
Original Articles
Information
Canadian Journal of Neurological Sciences , Volume 44 , Issue 4 , July 2017 , pp. 397 - 403
Copyright
Copyright © The Canadian Journal of Neurological Sciences Inc. 2017 

Introduction

Stroke is the third leading cause of death 1 and a leading cause of hospitalization in Canada, accounting for 16.9% of total hospitalizations. 2 Approximately one third of strokes are preceded by one or more transient ischemic attack (TIA). 3 However, unlike acute stroke, many individuals with a TIA initially seek medical attention from their primary care provider,Reference Goldstein, Bian, Samsa, Bonito, Lux and Matchar 4 - Reference Roebers, Wagner, Ritter, Dornbach, Wahle and Heuschmann 7 and the majority are not hospitalized.Reference Goldstein, Bian, Samsa, Bonito, Lux and Matchar 4 Population-based studies have reported that <10% of individuals with TIA present to the emergency department,Reference Giles, Flossman and Rothwell 8 and a recent study in OntarioReference Hall, O’Callaghan and Bayley 9 showed that only 17% of TIA patients were admitted to hospital. As a result, hospitalization data underestimate TIA occurrence, with previous work suggesting that population-based occurrence is underestimated by >66%.Reference Tu, Wang and Young 10

Although the Canadian Institute of Health Information (CIHI) discharge abstract database has previously been validated for the identification of stroke and TIAs.Reference Andrade and Harrold 11 - Reference Benesch, Witter and Wilder 15 few studies have examined the accuracy of physician billing records for identifying cases of TIA. As primary care diagnoses of TIA are subject to poor diagnostic accuracy,Reference Benesch, Witter and Wilder 15 - Reference Piriyawat, Smajsová and Smith 17 prior studies have shown reduced sensitivity for TIA case definitions involving physician billing data compared to those using hospitalization data alone.Reference Tu, Wang and Young 10 , Reference Newton, Wagner and Ramsey 18 , Reference Agrawal, Johnston, Wu, Sidney and Fullerton 19 However, the majority of these studies evaluated single-case definitions for TIA and did not involve population-based samples. Specifically, two previous studies used outpatient data to identify TIA as a comorbid diagnosis to diabetesReference Newton, Wagner and Ramsey 18 and atrial fibrillation,Reference Brophy, Snyder, Gaehde, Ives, Gagnon and Fiore 20 and two others assessed the ascertainment of TIA in paediatric patient samples.Reference Agrawal, Johnston, Wu, Sidney and Fullerton 19 , Reference Golomb, Garg, Saha and Williams 21 One recent studyReference Tu, Wang and Young 10 evaluated multiple case definitions for TIA from a sample of the general adult population and, in contrast to other work, reported improved sensitivity with the addition of physician billing data, but this study was cross-sectional and limited to billing data from volunteering family practice physicians.

The purpose of the present study was to evaluate the diagnostic accuracy of population-based physician billing administrative data for the identification of TIA cases. Using a validated algorithm for identifying ischemic stroke from hospital discharge data, we identified a cohort of individuals with stroke with a recent TIA as the reference standard and compared multiple algorithms for TIA case ascertainment from physician billing data in this cohort. Given the number of individuals evaluated and diagnosed with TIA in primary care settings, improving the ability to accurately identify these individuals has important implications for estimating the population-based occurrence of TIA, evaluating the impact of TIA on health outcomes, and for studies of health service utilization and health systems planning in this population.

Materials and Methods

Data Sources

The data for this study were obtained from the British Columbia (BC) Ministry of Health, via Population Data BC, and included linked records from: (1) the CIHI Hospital Discharge file, containing data on all hospital discharges, transfers and deaths from acute care hospitals in BC, excluding emergency department codes; and (2) the Medical Service Plan (MSP) Master Payment Information file, containing data on all medically necessary services provided on a fee-for-service basis by practitioners to individuals covered by MSP from 1992 to 2007. 22 During the period of this data extract, approximately 90% of physicians in BC operated on a fee-for-service payment basis. 23 The data in this extract were linkable across databases using individual‐specific personal health numbers assigned to all permanent residents of BC, with the exception of individuals covered by federal healthcare funding, including First Nations, veterans and those residing in federal detention institutions.

TIA Billing Practices

Prior to defining the base population and test algorithms for the identification of TIA, interviews with clinicians practicing in BC—including a stroke neurologist and a general practitioner (GP)—and administrative personnel—including billing clerks from a random sample of 20 general practice offices across BC and billing administrators from the Vancouver Coastal Health, Northern Health and Interior Health Authorities—were conducted to characterize practices used in the generation of diagnostic records within these administrative databases and identify common clinical trajectories for TIA patients across different care settings.

Based on these interviews, it was determined that, across health authorities, the use of International Classification of Disease (ICD) versions 9 and 10 codes (ICD–9 435.x or ICD–10 G45.x) in the primary diagnosis position was the primary method of coding hospital admissions for TIA. Primary care providers reported the use of ICD–9 code 435 or ICD–10 code G45 to capture physician encounters with TIA, and the majority also indicated that suspected events would be coded as TIA at the initial encounter and then, if possible, confirmed at the time of a follow-up encounter. The main clinical trajectories for individuals with TIA presenting to their primary care provider were identified as: (1) follow-up with the same GP within 60 to 90 days of the initial encounter, (2) referral to a specialist neurologist or cardiologist and (3) referral to a hospital. This information was used to inform the development of algorithms for TIA case ascertainment evaluated in the present study.

Base Population

The base population included all residents of BC eligible for health services with a diagnosis of TIA during the study period. TIA diagnoses were identified using either: (1) ICD–9 code 435.x or ICD–10 code G45.x in the primary diagnosis position from the hospital discharge abstract database, or (2) ICD–9 code 435 or ICD–10 code G45, with a corresponding specialist code for general practice [00], neurology [02] or cardiology [26] from the physician billing database (Table 1).

Table 1 Description of cohorts for base population, reference standard and administrative data algorithms for the identification of TIA diagnoses

GP=general practitioner; SP=specialist practitioner; CIHI=Canadian Institute of Health Information; ICD–9/10=International Classification of Diseases, versions 9 and 10.

Reference Standard

The reference standard cohort consisted of all individuals with a hospitalization for ischemic stroke and either a hospitalization or primary care diagnosis of TIA within 90 days prior to the stroke. Although rates of stroke after TIA are highly variable,Reference Giles and Rothwell 24 as recurrent stroke is more likely to occur after definite TIA than in individuals with transient neurological episodes that mimic TIA,Reference Amort, Fluri and Schäfer 25 short-term (<90 day) stroke recurrence was used as a criterion to retrospectively identify individuals with “true” TIA for the reference standard. Hospital admissions for stroke were captured using a previously validated algorithm for ischemic stroke from the discharge abstract database, involving ICD–9 codes 434.x and 436.x or ICD–10 codes I63.x and I64.x in the primary diagnosis position.Reference Mayo, Danys, Carlton and Scott 13 , Reference Kokotailo and Hill 14 This cohort was then restricted to cases that had a diagnosis of TIA within 90 days prior to the stroke diagnosis, using the above-described coding for TIA from either the physician billing (single code) or discharge abstract databases (inpatient codes in the primary diagnosis position) to form the reference standard cohort (Table 1).

TIA Algorithms

Several different administrative data algorithms for the identification of TIA were evaluated. For comparability to prior studies, a previously validated algorithm using ICD–9 code 435.x or ICD–10 code G45.x as the primary or most responsible diagnosis was included to capture TIA cases from CIHI hospital discharge records.Reference Kokotailo and Hill 14 , Reference Heckbert, Kooperberg and Safford 26 Based on the potential for bias associated with TIA diagnoses from single primary care encounters, all algorithms for the ascertainment of TIA cases from physician billing data consisted of diagnostic records from two consecutive physician encounters within 90 days and included the following combinations of potential encounters: (1) a GP encounter with a GP follow-up, (2) a GP encounter and specialist referral follow-up and (3) a GP encounter with a hospitalization (Table 1).

Analyses

Measures of diagnostic accuracy, including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive and negative likelihood ratios, overall accuracy and associated 95% confidence intervals (CI 95%) were calculated to compare each algorithm to the reference standard.Reference Akobeng 27 These measures were obtained for each individual algorithm, as well as for pre-specified combinations of algorithms, where cases captured from the different physician billing algorithms were added to the hospital discharge algorithm. Due to the potential for overlap in cases between the hospital discharge algorithm and the algorithm capturing TIA cases from an initial primary care encounter with a subsequent hospital admission, these two algorithms were not assessed in combination. Receiver-operating characteristic (ROC) analyses were conducted to compare area under the curve (AUC) values for each physician billing algorithm.Reference DeLong, DeLong and Clarke-Pearson 28 To compare the effect of the different case definitions on population-based estimates of TIA occurrence for the 3 million residents of BC over the study period, 22 we also estimated the annual TIA occurrence associated with each algorithm.

Results

A total of 102,492 individuals with a diagnostic record of TIA from the physicians’ billing or discharge abstract databases were identified for the base population. The reference cohort consisted of 3,110 individuals with an admission for ischemic stroke with a diagnosis of TIA within 90 days prior to stroke admission. A total of 21,284 cases were identified from the CIHI hospital discharge abstracts, 23,300 were identified from the algorithm capturing diagnoses from consecutive GP encounters, 3,931 were identified using the algorithm for a GP encounter with specialist referral, and 1,384 were identified using the algorithm with a GP visit with subsequent hospitalization for TIA (Table 1).

Results indicated that hospitalization records demonstrated high sensitivity for the identification of TIA cases 69.4% (CI 95%=69.1–69.8). By contrast, algorithms for TIA case ascertainment from physician billing data demonstrated reduced sensitivity, ranging from a very low sensitivity of 5.8% (CI 95%=5.7–6.1) for the algorithm capturing TIA from a GP encounter with specialist referral follow-up, to moderate sensitivity at 37.7% (CI 95%=37. 4–38.1) for the algorithm capturing TIA via two consecutive GP encounters. Sensitivity for algorithms combining physician billing data with cases identified via hospital discharge records was also poor, with these algorithms validating only 4% (CI 95%=4.3–4.6) and 12% (CI 95%=12.2–12.8) of cases, respectively (Table 2).

Table 2 Validation of administrative data algorithms for the ascertainment of TIA cases using hospital discharge records for individuals with ischemic stroke with prior TIA as a reference standard

GP=general practitioner; SP=specialist practitioner; CIHI=Canadian Institute of Health Information; CI 95%=95% confidence interval; PPV=positive predictive value; NPV=negative predictive value.

All physician billing data algorithms demonstrated high specificity for TIA case ascertainment, ranging from 76.6% (CI 95%=76.3–77.0) for the GP with specialist referral definition to 96.3% (CI 95%=96.1–96.5) for cases identified from an initial GP encounter with subsequent hospitalization (Table 2). As a result of the low to moderate sensitivity and high specificity observed across algorithms, the PPVs and likelihood ratios associated with each algorithm were also very low, with the algorithm for TIA cases ascertained from an initial GP diagnosis with a subsequent hospital admission demonstrating the highest PPV at 5.8% (CI 95%=4.9–5.3) (Table 2). False positive rates (1 – specificity) were comparable across physician billing algorithms and included approximately 20% of identified cases, with the exception of the algorithm capturing an initial GP diagnosis with a subsequent hospitalization for TIA, which had the highest specificity and identified few false positives (4%).

ROC analyses demonstrated that physician billing algorithms showed very poor discriminability, with AUC values ranging from 0.53 to 0.57 (Table 3). Although all algorithms tested in the present study showed similarly low classification accuracy, comparisons of observed AUC values revealed significant differences between algorithms for the classification of TIA (Table 3).

Table 3 Receiver-operating characteristic analysis of the discriminability of physician billing algorithms for the ascertainment of TIA

AUC=area under the curve; SE=standard error; CI 95%=95% confidence interval.

† Pairwise comparisons of AUC values for all physician billing algorithms compared to hospital-only definition, p<0.01.

For all algorithms, the population-based occurrence of TIA declined over time across the study period (1992–2007) (Figure 1). The average population-based occurrence of TIA associated with the reference standard was 0.08%. The average TIA occurrence using data from the CIHI discharge database alone was estimated at 0.5%. When algorithms for the identification of individuals with TIA from physician billing data were applied to the entire BC population across years, the algorithm involving two consecutive GP encounters showed an average estimated occurrence similar to the hospital discharge algorithm (0.6%), while algorithms capturing TIA cases from a GP encounter with specialist referral follow-up and a GP encounter with a subsequent hospitalization produced estimates similar to the reference standard, at 0.1 and 0.03%, respectively. When the algorithms for diagnoses based on a hospital discharge alone and two GP encounters were assessed in combination, the average population-based occurrence of TIA was estimated at 1.2% (Figure 1).

Figure 1 Impact of administrative data algorithm definitions on population-based estimates of TIA occurrence for the population of British Columbia from 1992–2007.

Discussion

The purpose of this study was to assess the diagnostic accuracy of multiple algorithms for the ascertainment of TIA from a longitudinal population-based physician billing administrative database. The observed accuracy of population-based physician billing data for the identification of TIA cases was poor, with algorithms demonstrating low to moderate sensitivity, low positive predictive values and area under the curve discriminability, and high false positive rates. However, when combined with a previously validated algorithm for TIA case ascertainment from hospital discharge data, this algorithm produced population-based estimates of annual TIA occurrence comparable to previous validation studies. These findings indicate that population-based physician billing data have insufficient sensitivity for identification of individuals with TIA, but algorithms capturing multiple GP encounters may be a useful addition to hospital discharge data for estimating population-based burden of TIA that includes patients diagnosed in primary care.

The results of this study are consistent with prior studies evaluating the use of outpatient physician billing records to ascertain TIA, where measures of sensitivity ranged from 33 to 53%.Reference Tu, Wang and Young 10 , Reference Newton, Wagner and Ramsey 18 , Reference Agrawal, Johnston, Wu, Sidney and Fullerton 19 The optimal algorithm for identification of TIA evaluated in the present study involved a TIA diagnosis from an initial GP encounter with a GP follow-up within 90 days. Similar to a prior study assessing the accuracy of identifying patients with stroke and TIA from participating family practices in Ontario,Reference Tu, Wang and Young 10 the algorithm of consecutive GP visits tested in the present study accurately validated 38% of TIA cases compared to the reference standard. However, these values are substantially lower than the reported discriminative utility for triaging TIA using clinical tools, such as the ABCD 3 and particularly the ABCD 3 –I score.Reference Merwick, Albers and Amarenco 29 Unlike prior work, the addition of physician billing records to the hospital discharge algorithm also did not improve the sensitivity of this case definition.Reference Tu, Wang and Young 10 Algorithms involving either specialist referral or hospital admission subsequent to the initial GP encounter assessed in the present study also showed low sensitivity (≤11%) for the identification of TIA. These findings may reflect management strategies for TIA in the primary care setting. Specifically, prior audits of primary care practices in the United States and abroad have reported low specialist and hospital referral rates associated with the primary care evaluation of TIA and a preference among GPs for the short-term outpatient management of TIA,Reference Goldstein, Bian, Samsa, Bonito, Lux and Matchar 4 , Reference Roebers, Wagner, Ritter, Dornbach, Wahle and Heuschmann 7 suggesting that TIA diagnoses are most likely to be accurately captured from algorithms for outpatient administrative data that include records from multiple visits within a short follow-up period, as observed in the present study.

Comparable to prior hospital-based studies, the sensitivity associated with an algorithm using only hospital discharge records to ascertain TIA in the present study was moderately high (69%).Reference Heckbert, Kooperberg and Safford 26 This finding confirmed the internal validity of using the criteria of stroke subsequent to TIA to define the reference standard. However, the PPVs associated with the physician billing algorithms evaluated in the present study were substantially lower than previously reported,Reference Newton, Wagner and Ramsey 18 , Reference Agrawal, Johnston, Wu, Sidney and Fullerton 19 and all algorithms showed poor AUC discriminability. A potential explanation for this finding relates to the definition for the reference standard employed in the present study. The occurrence of stroke after TIA in the base population used in our study was low (2.4%). As both PPV and AUC are dependent on the prevalence of the disease in the population being tested,Reference Altman and Bland 30 the use of a reference standard definition based on the criterion of stroke recurrent to TIA may have resulted in overestimation of cases detected by each of the physician billing algorithms, producing high false positive rates and lower estimates of PPV and AUC. For instance, although the algorithm that selected TIA cases using diagnostic records from two consecutive GP encounters showed the highest sensitivity (38%), the false positive rate for this algorithm was high (22%), resulting in a low PPV (5%) for this case definition. In addition, due to the high rates of false positivity across algorithms, significant differences in AUC observed between algorithms should be interpreted with caution.Reference Berrar and Flach 31 Despite these findings, as many individuals with TIA are diagnosed and managed by their primary care provider,Reference Goldstein, Bian, Samsa, Bonito, Lux and Matchar 4 , Reference Edlow, Kim, Pelletier and Camargo 32 case definitions that include individuals diagnosed with TIA in both hospital and outpatient settings are required for population-based research on health service utilization and resource planning. Thus, the use of clinical outcomes to decrease the potential for misclassification in the retrospective identification of TIA from administrative data records merits further study.

Notably, when the optimal physician billing algorithm from the present study was combined with an algorithm for identifying TIA from hospital discharge data, the average estimated annual occurrence of TIA over the study period was 1.2%—the same as that reported for a similar algorithm by Tu et al. (2013)Reference Tu, Wang and Young 10 (Figure 1). This finding suggests that the addition of cases ascertained from population-based physician billing data to hospital discharge records may be used to estimate TIA burden in the general population over time. One potential advantage of the algorithm assessed in the present study compared to the algorithm assessed by Tu et al. (2013) is the shorter follow-up window between family physician encounters (90 days vs. 1 year), decreasing the likelihood of capturing recurrent TIA events.

The strengths of this study included the use of longitudinal population-based data and the development of multiple case definitions to assess the accuracy of physician billing records for the identification of TIA. The use of information on billing practices and clinical trajectories from health professionals and billing administrators in different care settings for the development of comparison algorithms also increased the validity of these algorithms. However, the present study had several limitations. The use of conservative case definitions requiring two consecutive physician encounters for the identification of individuals with TIA may have contributed to the low observed sensitivities for physician billing algorithms in our study. In addition, the reference standard we used represented a high-risk subgroup of the TIA population (i.e., those who had a stroke recurrent to TIA). As rates of stroke recurrent to TIA vary substantially across studies, this narrow definition for the reference standard may have misrepresented the rates of TIA associated with each physician billing algorithm and limited our ability to validate their accuracy. Indeed, it is possible, given that the physician billing algorithms tested in the present study reflect measurement of the burden of TIA across different and changing clinical pathways, that no single method may be able to yield a “true” measurement of TIA.

Conclusions

Although many individuals with TIA are diagnosed and managed in the primary care setting and do not undergo hospitalization, there are many challenges for accurately identifying TIA cases from physician billing data. Our study was novel in the use of longitudinal population-based data from physician encounters to develop and test multiple case definitions based on clinical trajectories of patients presenting with TIA. The findings confirmed that physician billing records have insufficient sensitivity for the classification of TIA. However, the evaluation of these algorithms also revealed that these data may be used in combination with hospital discharge data to improve the estimation of population-based trends in TIA occurrence. Although there may be no single method for capturing “true” cases of TIA, given the importance of identifying diagnoses from different care settings for population-based health services research, alternative approaches for the identification of outpatient cases of TIA are required.

Funding

This research was supported by a doctoral award from the Canadian Institutes of Health Research to JDE.

Disclosures

Jodi Edwards has the following disclosure. Canadian Institutes of Health Research to JDE: award recipient, financial support. Mieke Koehoorn, Lara Boyd, Boris Sobolev and Adrian Levy do not have anything to disclose.

References

1. Heart and Stroke Foundation of Canada. Stroke Statistics; 2008.Google Scholar
2. Public Health Agency of Canada. Tracking Heart Disease and Stroke in Canada; 2009. Available at: http://www.phac-aspc.gc.ca/publicat/2009/cvd-avc/index-eng.php.Google Scholar
3. Canadian Stroke Network. The Quality of Stroke Care in Canada; 2011. Available at: http://canadianstrokenetwork.ca/en/wp-content/uploads/2014/08/QoSC-EN.pdf.Google Scholar
4. Goldstein, LB, Bian, J, Samsa, GP, Bonito, AJ, Lux, LJ, Matchar, DB. New transient ischemic attack and stroke: outpatient management by primary care physicians. Arch Intern Med. 2000;160(19):2941-2946.CrossRefGoogle ScholarPubMed
5. Chandratheva, A, Lasserson, DS, Geraghty, OC, Rothwell, PM. Population-based study of behavior immediately after transient ischemic attack and minor stroke in 1000 consecutive patients: lessons for public education. Stroke. 2010;41(6):1108-1114. Available at: http://stroke.ahajournals.org/content/41/6/1108.long/ CrossRefGoogle ScholarPubMed
6. Wester, P, Rådberg, J, Lundgren, B, Peltonen, M. Factors associated with delayed admission to hospital and in-hospital delays in acute stroke and TIA: a prospective, multicenter study. Seek-Medical-Attention-in-Time Study Group. Stroke. 1999;30(1):40-48. Available at: http://stroke.ahajournals.org/content/30/1/40.long.Google ScholarPubMed
7. Roebers, S, Wagner, M, Ritter, MA, Dornbach, F, Wahle, K, Heuschmann, PU. Attitudes and current practice of primary care physicians in acute stroke management. Stroke. 2007;38(4):1298-1303; Epub ahead of print Mar 1. Available at: http://stroke.ahajournals.org/content/38/4/1298.long.Google Scholar
8. Giles, MF, Flossman, E, Rothwell, PM. Patient behavior immediately after transient ischemic attack according to clinical characteristics, perception of the event, and predicted risk of stroke. Stroke. 2006;37(5):1254-1260; Epub ahead of print Mar 30. Available at: http://stroke.ahajournals.org/content/37/5/1254.long.CrossRefGoogle Scholar
9. Hall, R, O’Callaghan, C, Bayley, M. Ontario Stroke Evaluation Report 2010: Technical Report. Toronto: Institute for Clinical Evaluative Sciences; 2010. Available at: http://www.ices.on.ca/Publications/Atlases-and-Reports/2010/Ontario-Stroke-Evaluation-Technical-Report.Google Scholar
10. Tu, K, Wang, M, Young, J, et al. Validity of administrative data for identifying patients who have had a stroke or transient ischemic attack using EMRALD as a reference standard. Can J Cardiol. 2013;29(11):1388-1394; Epub ahead of print Sep 26.Google Scholar
11. Andrade, SE, Harrold, LR, et al. A systematic review of validated methods for identifying cerebrovascular accident or transient ischemic attack using administrative data. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 1):100-128.Google Scholar
12. Tirschwell, DL, Longstreth, WT Jr. Validating administrative data in stroke research. Stroke. 2002;33(10):2465-2470. Available at: http://stroke.ahajournals.org/content/33/10/2465.long.Google Scholar
13. Mayo, NE, Danys, I, Carlton, J, Scott, SC. Accuracy of hospital discharge coding for stroke. Can J Cardiol. 1993;9:590-595.Google Scholar
14. Kokotailo, RA, Hill, MD. Coding of stroke and stroke risk factors using international classification of diseases, revisions 9 and 10. Stroke. 2005;36(8):1776-1781. Available at: http://stroke.ahajournals.org/content/36/8/1776.long.Google Scholar
15. Benesch, C, Witter, DM Jr, Wilder, AL, et al. Inaccuracy of the International Classification of Diseases (ICD–9–CM) in identifying the diagnosis of ischemic cerebrovascular disease. Neurology. 1997;49(3):660-664.Google Scholar
16. Goldstein, LB. Accuracy of ICD–9–CM coding for the identification of patients with acute ischemic stroke: effect of modifier codes. Stroke. 1998;29(8):1602-1604. Available at: http://stroke.ahajournals.org/content/29/8/1602.long.CrossRefGoogle ScholarPubMed
17. Piriyawat, P, Smajsová, M, Smith, MA, et al. Comparison of active and passive surveillance for cerebrovascular disease: The Brain Attack Surveillance in Corpus Christi (BASIC) Project. Am J Epidemiol. 2002;156(11):1062-1069. Available at: http://aje.oxfordjournals.org/content/156/11/1062.long.Google Scholar
18. Newton, KM, Wagner, EH, Ramsey, SD, et al. The use of automated data to identify complications and comorbidities of diabetes: a validation study. J Clin Epidemiol. 1999;52(3):199-207.CrossRefGoogle ScholarPubMed
19. Agrawal, N, Johnston, SC, Wu, YW, Sidney, S, Fullerton, HJ. Imaging data reveal a higher pediatric stroke incidence than prior US estimates. Stroke. 2009;40(11):3415-3421; Epub ahead of print Sep 17. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3387270/.Google Scholar
20. Brophy, MT, Snyder, KE, Gaehde, S, Ives, C, Gagnon, D, Fiore, LD. Anticoagulant use for atrial fibrillation in the elderly. J Am Geriatr Soc. 2004;52(7):1151-1156.CrossRefGoogle ScholarPubMed
21. Golomb, MR, Garg, BP, Saha, C, Williams, LS. Accuracy and yield of ICD–9 codes for identifying children with ischemic stroke. Neurology. 2006;67(11):2053-2055.CrossRefGoogle ScholarPubMed
22. Population Data British Columbia; Vancouver: PopData; 2016. Available at: www.popdata.bc.ca.Google Scholar
23. Canadian Institute for Health Information. Average Payment per Physician Report: Fee-for-Service Physicians in Canada 2004–2005. Ottawa: Government of Canada; 2006.Google Scholar
24. Giles, MF, Rothwell, PM. Risk of stroke early after transient ischaemic attack: a systematic review and meta-analysis. Lancet Neurol. 2007;6(12):1063-1072; Epub ahead of print Nov 13.Google Scholar
25. Amort, M, Fluri, F, Schäfer, J, et al. Transient ischemic attack versus transient ischemic attack mimics: frequency, clinical characteristics and outcome. Cerebrovasc Dis. 2011;32(1):57-64.Google Scholar
26. Heckbert, SR, Kooperberg, C, Safford, MM, et al. Comparison of self-report, hospital discharge codes, and adjudication of cardiovascular events in the Women’s Health Initiative. Am J Epidemiol. 2004;160(12):1152-1158. Available at: http://aje.oxfordjournals.org/content/160/12/1152.full.pdf+html.CrossRefGoogle ScholarPubMed
27. Akobeng, AK. Understanding diagnostic tests, 1: sensitivity, specificity and predictive values. Acta Paediatr. 2007;96(3):338-341.Google Scholar
28. DeLong, ER, DeLong, DM, Clarke-Pearson, DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837-845.Google Scholar
29. Merwick, A, Albers, GW, Amarenco, P, et al. Addition of brain and carotid imaging to the ABCD(2) score to identify patients at early risk of stroke after transient ischaemic attack: a multicentre observational study. Lancet Neurol. 2010;9(11):1060-1069; Epub ahead of print Oct 11.CrossRefGoogle Scholar
30. Altman, DG, Bland, JM. Diagnostic tests 2: predictive values. BMJ. 1994;309(6947):102. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2540558/.Google Scholar
31. Berrar, D, Flach, P. Caveats and pitfalls of ROC analysis in clinical microarray research (and how to avoid them). Brief Bioinform. 2012;13(1):83-97. Available at: http://bib.oxfordjournals.org/content/13/1/83.full.pdf+html.CrossRefGoogle Scholar
32. Edlow, JA, Kim, S, Pelletier, AJ, Camargo, CA Jr. National study on emergency department visits for transient ischemic attack, 1992–2001. Acad Emerg Med. 2006;13(6):666-672; Epub ahead of print Apr 11.Google Scholar
Figure 0

Table 1 Description of cohorts for base population, reference standard and administrative data algorithms for the identification of TIA diagnoses

Figure 1

Table 2 Validation of administrative data algorithms for the ascertainment of TIA cases using hospital discharge records for individuals with ischemic stroke with prior TIA as a reference standard

Figure 2

Table 3 Receiver-operating characteristic analysis of the discriminability of physician billing algorithms for the ascertainment of TIA

Figure 3

Figure 1 Impact of administrative data algorithm definitions on population-based estimates of TIA occurrence for the population of British Columbia from 1992–2007.