This paper assesses the influence of heat and drought stress on the economic performance of the European dairy sector. Climatic data from the Gridded Agro-Meteorological data in Europe were combined with dairy enterprise data from the Farm Accountancy Data Network, resulting in a data set of 4412 farms in 22 European Union (EU) countries over the period 2007–2013. Since the performance of dairy farms is influenced by the context in which they operate, farms were grouped into areas representing similar climatic conditions through the use of a latent class analysis. Technical efficiency (TE) and economic downside risk were used as performance indicators against which the effect of climatic stress factors was evaluated. TE was estimated using a ‘true-fixed’ effect stochastic frontier model. Economic downside risk was based on gross margin deviations. Regression analysis suggests a significant negative effect of drought and heat stress on both TE and the downside gross margin difference in most climatic classes, with few exceptions. Results imply that both drought and heat stress-related issues need to be considered when designing adaptation strategies to address threats to the economic performance of the EU dairy sector.