Hostname: page-component-586b7cd67f-t8hqh Total loading time: 0 Render date: 2024-11-26T00:23:02.698Z Has data issue: false hasContentIssue false
  • English
  • Français

Development, implementation and reliability assessment of an emergency physician performance evaluation tool

Published online by Cambridge University Press:  21 May 2015

Jeremy Etherington ,
Grant Innes ,
James Christenson ,
Jonathan Berkowitz ,
Robert Chamberlain ,
Ross Berringer  and
Cosmas Leung
Jeremy Etherington*
Affiliation:
Department of Emergency Medicine, Providence Health Care, St. Paul’s Hospital Site, Vancouver, BC
Grant Innes
Affiliation:
Department of Emergency Medicine, Providence Health Care, St. Paul’s Hospital Site, Vancouver, BC
James Christenson
Affiliation:
Department of Emergency Medicine, Providence Health Care, St. Paul’s Hospital Site, Vancouver, BC
Jonathan Berkowitz
Affiliation:
Department of Emergency Medicine, Providence Health Care, St. Paul’s Hospital Site, Vancouver, BC
Robert Chamberlain
Affiliation:
Department of Emergency Medicine, Providence Health Care, St. Paul’s Hospital Site, Vancouver, BC
Ross Berringer
Affiliation:
Department of Emergency Medicine, Providence Health Care, St. Paul’s Hospital Site, Vancouver, BC
Cosmas Leung
Affiliation:
Department of Emergency Medicine, Providence Health Care, St. Paul’s Hospital Site, Vancouver, BC
*
Department of Emergency Medicine, St. Paul’s Hospital, 1081 Burrard St., Vancouver BC V6Z 1Y6; 604 806–8480, fax 604 806–8488 [email protected]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Evaluation of physician practice is necessary, both to provide feedback for self-improvement and to guide department heads during yearly evaluations.

Objective:

To develop and implement a peer-based performance evaluation tool and to measure reliability and physician satisfaction.

Methods:

Each emergency physician in an urban emergency department evaluated their peers by completing a survey consisting of 21 questions on effectiveness in 4 categories: clinical practice, interaction with coworkers and the public, nonclinical departmental responsibilities, and academic activities. A sample of emergency nurses evaluated each emergency physician on a subset of 5 of the questions. Factor analysis was used to assess the reliability of the questions and categories. Intra-class correlation coefficients were calculated to determine inter-rater reliability. After receiving their peer evaluations, each physician rated the process’s usefulness to the individual and the department.

Results:

225 surveys were completed on 16 physicians. Factor analysis did not distinguish the nonclinical and academic categories as distinct; therefore, the survey questions fell into 3 domains, rather than the 4 hypothesized. The overall intra-class correlation coefficient was 0.43 for emergency physicians, indicating moderate, but far from perfect, agreement. This suggests that variability exists between physician evaluators, and that multiple reviewers are probably required to provide a balanced physician evaluation. The intra-class correlation coefficient for emergency nurses was 0.11, suggesting poor reliability. Overall, 11 of 15 physicians reported the process valuable or mostly valuable, 3 of 15 were unsure and 1 of 15 reported that the process was definitely not valuable.

Conclusion:

Physician evaluation by a single individual is probably unreliable. A useful physician peer evaluation tool can be developed. Most physicians view a personalized, broad-based, confidential peer review as valuable.

Keywords

peer reviewphysician assessmentperformance evaluationphysician competence
Type
ED Administration • Administration Du DU
Information
Canadian Journal of Emergency Medicine , Volume 2 , Issue 4 , October 2000 , pp. 237 - 245
Copyright
Copyright © Canadian Association of Emergency Physicians 2000

References

1.Davis, DA., Norman, GR., Painvin, A., Lindsay, E., Ragbeer, MS., Rath, D.Attempting to ensure physician competence. JAMA 1990;263:2041–2.Google Scholar
2.Langsley, DG.Medical competence and performance assessment: a new era. JAMA 1991;266:977–80.CrossRefGoogle ScholarPubMed
3.Nash, DB., Markson, LE., Howell, S., Hildreth, EA.Evaluating the competence of physicians in practice: from peer review to performance assessment. Acad Med 1993;68:519–22.CrossRefGoogle ScholarPubMed
4.Ramsey, PG., Wenrich, MD., Carline, JD, Inui, TS., Larson, EB, LoGerfo, JP.Use of peer ratings to evaluate physician performance. JAMA 1993;269:1655–60.Google Scholar
5.Norman, GR., Davis, DA., Lamb, S., Hanna, E., Caulford, P., Kaigas, T.Competency assessment of primary care physicians as part of a peer review program. JAMA 1993;270:1046–51.Google Scholar
6.McAuley, RG., Henderson, HW.Results of the peer assessment program of the College of Physicians and Surgeons of Ontario. CMAJ 1984;131:557–61.Google Scholar
7.Gilbert, G.Relating quality assurance to credentials and privileges. QRB 1984;May:130–5.Google Scholar
8.Page, GG., Bates, J., Dyer, SM., Vincent, DR., Bordage, G., Jacques, A., et al. Physician assessment and physician enhancement programs in Canada. CMAJ 1995;153:1723–8.Google Scholar
9.Bates, J., Page, G., Vincent, D., Dyer, S.The assessment and enhancement of clinical competence in British Columbia, Canada. Proceedings of the 6th Ottawa Conference on Medical Education, Certification, Licensure, and CME; 1995.Google Scholar
10.College of Physicians and Surgeons of British Columbia. 1st College Quarterly. The Association; Winter 1995.Google Scholar
11.Eliasoph, H., Ashdown, C.Development, testing and implementation of an emergency services methodology in Alberta. Healthcare Manage Forum 1995;8:31–7.Google Scholar
12.Norman, GR., Streiner, DL.Biostatistics: the bare essentials. St. Louis: Mosby–Year Book, Inc; 1994. p. 129–42.Google Scholar
13.Medical Practitioners Act, British Columbia. Revised Statutes of British Columbia, 1979: chap. 254. Available: http://bbs.qp.gov.bc.ca/bcstatsGoogle Scholar
14.Hospital Act, British Columbia. Revised Statutes of British Columbia, 1995. chap. 200. Available: http://bbs.qp.gov.bc.ca/bcstatsGoogle Scholar
15.Granting of Privileges–Guidelines for Hospital Trustees, Administration and Medical Staff: Part III, Appointment/Reappointment Process. BC Medical Association, College of Physicians and Surgeons of BC and BC Hospitals Association: joint publication; June 1991.Google Scholar
16.Cruess, SR.Medical department heads. In: Gelman, DD, editor. The CMA guide to medical administration in Canadian hospitals. Ottawa, 1996. E3.3–3.4.Google Scholar