Emergency department (ED) triage prioritizes patients based on urgency of care, and the Canadian Triage and Acuity Scale (CTAS) is the national standard. We describe the inter-rater agreement and manual overrides of nurses using a CTAS-compliant web-based triage tool (eTRIAGE) for 2 different intensities of staff training.

This prospective study was conducted in an urban tertiary care ED. In phase 1, eTRIAGE was deployed after a 3-hour training course for 24 triage nurses who were asked to share this knowledge during regular triage shifts with colleagues who had not received training (n = 77). In phase 2, a targeted group of 8 triage nurses underwent further training with eTRIAGE. In each phase, patients were assessed first by the duty triage nurse and then by a blinded independent study nurse, both using eTRIAGE. Inter-rater agreement was calculated using kappa (weighted κ) statistics.

In phase 1, 569 patients were enrolled with 513 (90.2%) complete records; 577 patients were enrolled in phase 2 with 555 (96.2%) complete records. Inter-rater agreement during phase 1 was moderate (weighted κ = 0.55; 95% confidence interval [CI] 0.49–0.62); agreement improved in phase 2 (weighted κ = 0.65; 95% CI 0.60–0.70). Manual overrides of eTRIAGE scores were infrequent (approximately 10%) during both periods.

Agreement between study nurses and duty triage nurses, both using eTRIAGE, was moderate to good, with a trend toward improvement with additional training. Triage overrides were infrequent. Continued attempts to refine the triage process and training appear warranted.