Background: Neurology Residency training in Canada is transitioning to competence based medical education (CBME) in July 2020 and the Royal College Neurology Specialty Committee has identified “providing consultation for and managing patients at outlying centers,” to be an entrustable professional activity (EPA). At Western, neurology telephone consultations, from outlying centres, are attended by both the resident and the staff Neurologist. This scenario provides the ideal situation for direct observation and immediate formative feedback. The resident’s performance is assessed using the ’TeleTool’ which utilizes an entrustment scale and has a short narrative portion. Methods: This mixed methods study aims to determine the reliability and validity of the ’TeleTool’ in assessing the performance of the telephone consultation by senior neurology residents. Informed consent was obtained from residents (9) and staff (7) involved. Scores on the entrustment scale and narrative comments were analysed. Results: Information on 30 encounters (involving 9 residents) was collected. TeleTool results demonstrated higher entrustment scores in PGY4 and PGY5 levels. Overall, ratings were consistent across the 7 consultants assessors. Conclusions: The TeleTool was reliable and valid in assessing competence in the telephone consultation and will be a useful tool for assessment of this EPA.