Global Health Security Index (GHSI) and Joint External Evaluation (JEE) are two well-known health security and related capability indices. We hypothesised that countries with higher GHSI or JEE scores would have detected their first COVID-19 case earlier, and would experience lower mortality outcome compared to countries with lower scores. We evaluated the effectiveness of GHSI and JEE in predicting countries' COVID-19 detection response times and mortality outcome (deaths/million). We used two different outcomes for the evaluation: (i) detection response time, the duration of time to the first confirmed case detection (from 31st December 2019 to 20th February 2020 when every country's first case was linked to travel from China) and (ii) mortality outcome (deaths/million) until 11th March and 1st July 2020, respectively. We interpreted the detection response time alongside previously published relative risk of the importation of COVID-19 cases from China. We performed multiple linear regression and negative binomial regression analysis to evaluate how these indices predicted the actual outcome. The two indices, GHSI and JEE were strongly correlated (r = 0.82), indicating a good agreement between them. However, both GHSI (r = 0.31) and JEE (r = 0.37) had a poor correlation with countries' COVID-19–related mortality outcome. Higher risk of importation of COVID-19 from China for a given country was negatively correlated with the time taken to detect the first case in that country (adjusted R2 = 0.63–0.66), while the GHSI and JEE had minimal predictive value. In the negative binomial regression model, countries' mortality outcome was strongly predicted by the percentage of the population aged 65 and above (incidence rate ratio (IRR): 1.10 (95% confidence interval (CI): 1.01–1.21) while overall GHSI score (IRR: 1.01 (95% CI: 0.98–1.01)) and JEE (IRR: 0.99 (95% CI: 0.96–1.02)) were not significant predictors. GHSI and JEE had lower predictive value for detection response time and mortality outcome due to COVID-19. We suggest introduction of a population healthiness parameter, to address demographic and comorbidity vulnerabilities, and reappraisal of the ranking system and methods used to obtain the index based on experience gained from this pandemic.