Background: Brain metastases are frequently seen in neurosurgical practice. Standardised criteria are created to better classify these common pathologies in research studies. This study’s goal was to evaluate RANO-BM criteria’s current thresholds in a cohort of patients with brain metastases managed by SRS. Methods: We performed a retrospective metastasis-level analysis of patients treated with SRS for brain metastases. The data collected included cohort demographics, metastases characteristics, outcomes, and the rate of true positives, false negatives, true negatives and false positives as defined by RANO-BM criteria at last follow-up before second SRS. Results: 251 metastases in 50 patients were included in the analysis. RANO-BM criteria using current thresholds yielded a sensitivity of 38%, a specificity of 95%, a positive predictive value of 71% and a negative predictive value of 84%. Modified RANO-BM criteria using absolute diameter differences of 2.5 mm yielded a sensitivity of 83%, a specificity of 87%, a positive predictive value of 67% and a negative predictive value of 94%. Pseudoprogression occurred significantly earlier than tumor progression, with a median time of onset of 6.9 months and 12.1 months respectively. Conclusions: Current RANO-BM criteria unreliably identifies clinically relevant tumor progression, but are useful in assessing diameter increases caused by tumor progression and pseudoprogression.