Impact Statement

Reducing carbon emissions in residential heating is a major concern until the mid-21st century. To estimate the annual electricity consumption of future retrofitted heat pumps that replace gas furnaces, we need to know the expected seasonal performance of these heat pumps. With the availability of more and more smart meter data, we propose and evaluate a new method to estimate the seasonal performance of future heat pumps. This method involves converting current annual heat consumption measures, based on annual gas consumption, to the electricity consumption that a retrofitted heat pump would require. The novelty of this method is the use of either the Jensen–Shannon Divergence (JSD) as an instance of a distance measure or the Kolmogorov–Smirnov (KS) test as an instance of a statistical test on an unpaired dataset of annual gas and heat pump electricity consumption values.

2. Related Work

Real measured gas consumption data is used by Kolter and Ferreira (Reference Kolter and Ferreira2011) to predict and visualize the heat demand on the building level. Ko (Reference Ko2013) also utilizes heat data measured on the building level to provide an statistical analysis of heat and energy usage in the urban sector, including an overview of techniques for modeling urban energy usage. When hourly heat demand data for a building is available alongside weather data, the model proposed by Lumbreras et al. (Reference Lumbreras, Garay-Martinez, Arregi, Martin-Escudero, Diarce, Raud and Hagu2022) can be employed for long-term, hourly heat demand predictions.

Several models exist that estimate either the Coefficient of Performance (COP) (Shin and Cho, Reference Shin and Cho2021; Priarone et al., Reference Priarone, Silenzi and Fossa2020) or the SPF (Nouvel et al., Reference Nouvel, Cotrado and Pietruschka2015) based on theoretical computations or laboratory measures. In the latter, for instance, the authors present a model for estimating the SPF of a highly simplified sample building located in various regions of Europe, illustrating how location-specific climatic conditions can affect the SPF. This variation in SPF across different regions is also investigated by Rossi di Schio et al. (Reference Rossi di Schio, Ballerini, Dongellini and Valdiserri2021), who demonstrate that air-source heat pumps experience changes in performance due to the varying number of defrosting cycles required in different climates. Meanwhile, field studies on the electricity consumption and SPF of real-world heat pump installations, such as those by Huchtemann and Müller (Reference Huchtemann and Müller2012), Miara et al. (Reference Miara, Russ, Günther, Kramer and Henning2014) or Ruschenburg et al. (Reference Ruschenburg, Ćutić and Herkel2014), reveal notable differences in the SPF from the theoretical computed values. Finally, Carroll et al. (Reference Carroll, Chesser and Lyons2020) provide a detailed review of field studies on air-source heat pump efficiencies, highlighting notable variations in the resulting SPF values depending on the region under consideration.

Given that the SPF is heavily influenced by both the geographic location and the specific year due to weather fluctuations, and real-world SPF values often differ notably from theoretical calculations, an accurate estimation of the SPF is crucial for effective energy planning. Computing the SPF in a field trial requires metering the actual heat produced by a heat pump, which can be labor-intensive and costly. Therefore, using smart meter data to estimate the SPF appears to be a promising approach for computing average SPF values at the city level, especially since the SPF can differ from one region to another, as mentioned in the previous paragraph. So far, only a few recent works have used real gas consumption data, assuming these gas furnaces should be replaced by heat pumps. For example, Lechowicz et al. (Reference Lechowicz, Bashir, Wamburu, Hajiesmaili and Sheno2023) use gas consumption data of a city to compute the residential energy demand on a household level using an assumed COP. They focus on selecting regions of the city where heat pumps should replace all existing gas furnaces to save maintenance costs at the gas grid level. Wamburu et al. (Reference Wamburu, Bashir, Irwin and Shenoy2022) present an analysis of gas consumption in a residential area based on real data. Additionally, they propose an optimization framework for reducing carbon dioxide emissions. Also, other publications focusing on the effects of a heat pump retrofit use assumed values for the COP or SPF that are not aligned to the concrete weather conditions of a year like Walker et al. (Reference Walker, Less and Casquero-Modrego2022) or Jenkins et al. (Reference Jenkins, Tucker and Rawlings2009). The disadvantage of using SPF values that are not aligned with the actual weather conditions occurs in cold winter. There, the actual performance is overestimated, leading to an underestimation of the required electricity consumption of the heat pumps (Petrović and Karlsson, Reference Petrović and Karlsson2016). Thus, the estimation of the SPF that is aligned with the actual annual weather and consumption behavior patterns is an important topic.

3. Methodology

Let $ {H}^{Gas} $ be the set of buildings with existing gas furnaces that should be replaced by heat pumps and $ {H}^{HP} $ be the set of buildings with already existing heat pumps. Based on the existing gas consumption data of year $ j $ , we can compute the thermal energy $ {Q}_j(h) $ produced by a gas furnace in a building $ h\in {H}^{Gas} $ similar to Lechowicz et al. (Reference Lechowicz, Bashir, Wamburu, Hajiesmaili and Sheno2023) using

(1) $$ {Q}_j(h)={R}_j(h)\cdot z\cdot \nu \cdot {\eta}_g $$

where $ {R}_j(h) $ is the volume of gas consumed in year $ j $ , $ z $ is the pressure factor, $ \nu $ is the heating value of the gas, and $ {\eta}_g $ is the efficiency of the gas furnace. In this paper, we expect that the heat consumption follows the heat demand.

The heat consumption of a building is highly dependent on its individual insulation level. The insulation level between gas and heat pump heated buildings might be different, due to two reasons. Firstly, it is possible that a building’s insulation level will be improved if a gas furnace is replaced by a heat pump, as it is reported in a field trial by Kelly and Cockroft (Reference Kelly and Cockroft2011). Secondly, new buildings are generally better insulated than older buildings. However, it can be assumed that gas heating systems tend to be installed in older buildings, as heat pumps have started to become widespread in private buildings in the last decade. Therefore, we introduce the parameter $ \gamma (h) $ , which denotes the heat demand reduction that goes along with the replacement of the gas furnace in building $ h\in {H}^{Gas} $ . The heat consumption $ {Q}_j^{\Delta}(h) $ after the retrofit is thus given by

(2) $$ {Q}_j^{\Delta}(h)={Q}_j(h)\cdot \left(1-\gamma (h)\right) $$

According to Hailu (Reference Hailu, Borge-Diez and Rosales-Asensio2021), we define the SPF of a heat pump installed in building $ h\in {H}^{HP} $ in the year $ j $ as

(3) $$ {\mathrm{SPF}}_j(h)=\frac{Q_j^{HP}(h)}{E_j(h)} $$

where $ {E}_j(h) $ denotes the electrical energy consumed by the heat pump in building $ h $ in year $ j $ and $ {Q}_j^{HP}(h) $ denotes the heat (i.e., thermal energy) produced by this heat pump in year $ j $ . We note that the considered heat pumps produce domestic hot water (but no cooling loads) during summer in our considered location.

The first assumption for this methodology is that the annual heat consumption of a building currently heated by a gas furnace does not change upon a heat pump retrofit except for contemporaneous building insulation retrofits, i.e., $ {Q}_j^{\Delta}(h)={Q}_j^{HP}(h) $ . We will discuss on this assumption in Section 5.2. Thus, we can compute the electrical energy that a retrofitted heat pump in building $ h\in {H}^{Gas} $ would consume by combining Equations (1), (2), and (3):

(4) $$ {E}_j(h)={Q}_j(h) \cdot \underset{=:\ {B}_j^{-1}(h)}{\underbrace{\frac{1-\gamma (h)}{{\mathrm{SPF}}_j(h)}}} $$

As mentioned in the Section 1, measuring the heat produced by a heat pump can be very challenging. Nevertheless, the electricity consumption of an existing heat pump can be metered much more quickly if a separate smart meter is installed. As a result, calculating the SPF for individual buildings becomes impossible without dedicated meters to measure the annual heat output of their heat pumps. However, as we have data on the electricity consumption of heat pumps on one hand and, on the other, data on the annual heat consumption of buildings from the same class from gas smart meter data, we compute the average SPF in the following by shifting our focus from the individual building level to all buildings. In the further course, the distribution of the annual heat consumption over all single- and two-family buildings might be denoted as $ {Q}_j $ . The heat consumption distribution of those only heated by a heat pump is denoted as $ {Q}_j^{HP} $ , with $ {Q}_j^{HP}\subset {Q}_j $ , and those only heated by a gas furnace are denoted by $ {Q}_j^{Gas} $ , with $ {Q}_j^{Gas}\subset {Q}_j $ .

Over the complete building stock of a city, the individual annual heat consumption values typically show a high degree of variation, as the annual heat consumption of a residential building depends on multiple, often hidden parameters (Lim and Zhai, Reference Lim and Zhai2017). The most important parameters are as follows:

• • the weather conditions,

• • the building size (Kolter and Ferreira, Reference Kolter and Ferreira2011),

• • the household size and income (Sardianou, Reference Sardianou2008),

• • the user behavior and their preferred temperature (Delzendeh et al., Reference Delzendeh, Wu, Lee and Zhou2017),

• • misconfigurations of the heating system (Weigert et al., Reference Weigert, Hopf, Günther and Staake2022), and

• • the insulation standard.

This leads to a broad distribution of the actual heat consumption over the complete residential building stock in a given city (see for example Figure 1). Even though early publications reported less impact of the insulation on the building heat consumption, especially in the case of an increase of the insulation level during a retrofit (Capper and Scott, Reference Capper and Scott1982), more recent approaches report notable impacts dependent on the climatic conditions (Pan et al., Reference Pan, Chan, Deng and Lin2012).

Figure 1. Histogram of the annual gas (orange) and heat pump electricity consumption (green) from our dataset in 2020 in the upper row. Lower row: The scatter plot of the annual consumption values across all single- and two-family buildings plotted against the corresponding building size. The solid black line is the result of the linear regression.

The main idea of this work is to assume that the influence exerted by the above-listed parameters on the annual heat consumption of an individual building is independent of the specific heating system type. This assumption implies a uniform impact of these factors on energy consumption, regardless of the underlying heating system. In other words, the distribution of the annual heat consumption over all buildings heated with gas furnaces $ {Q}_j^{Gas} $ and heated with heat pumps $ {Q}_j^{HP} $ is assumed to be drawn from the same underlying distribution $ {Q}_j $ of all heat consumptions that only depends on the above-listed parameters but is independent of the heating system type, i.e.,

(5) $$ \mathrm{\forall}j\in J:\hskip2pt {Q}_j\sim {Q}_j^{HP}\sim {Q}_j^{Gas} $$

where $ J $ denotes the set of considered years and the tilde denotes the equality of two distributions. We note that we only consider heat pumps and gas furnaces as heating systems in this paper. Nevertheless, the main assumption does not imply that the above-listed parameters would not have any impact on the residential heat consumption or the efficiency of heat pumps itself. As reported in related works like Huchtemann and Müller (Reference Huchtemann and Müller2012), the concrete weather conditions in a given year impact the seasonal performance of heat pumps notably. Since the other parameters mentioned above that affect heat consumption in residential buildings do not change frequently between years for a given building, it can be assumed that the main cause of varying heat pump performance is the changing weather conditions.

Let us further assume that the SPF divided by $ 1-\gamma (h) $ (i.e., $ {B}_j^{-1}(h) $ in Eq. (4)) is independent of the actual heat consumption $ {Q}_j(h) $ of an arbitrary building $ h $ . Finally, it can be assumed that the influence of the weather on the SPF affects all heat pumps equally. This implies that $ {Q}_j^{HP}={E}_j\cdot {B}_j $ holds with $ {E}_j $ denoting the distribution of the metered annual heat pump electricity consumption and $ {B}_j $ denotes the average value of $ {B}_j(h) $ overall existing heat pumps in year $ j $ with $ {B}_j $ only influenced by the changing weather conditions. Then, Eq. (6) follows from Eq. (5) for the distributions computed over the obtained dataset:

(6) $$ \mathrm{\forall}j\in J:\hskip2pt {Q}_j^{HP}={E}_j\cdot {B}_j\sim {Q}_j^{Gas} $$

Obtaining statistically significant similarities between $ {E}_j\cdot {B}_j $ and $ {Q}_j^{Gas} $ for at least one $ {B}_j $ for every considered year $ j $ provides evidence of the assumptions’ validity (see Section 4 and Table 1).

Table 1. Results (first section), changes of the mean value of the input data for different years (center section) and exogenous variables (last section) for the four evaluated years

¹ p-value of KS test

² Compared to the reference year 2020

³ The winter is defined as the period from 01.01. until 20.02. and 01.11. until 31.12. of each year

3.1. Find optimal B

To get the value of $ B $ where both distributions of Eq. (6) overlap best, we propose two different methods. First, we select the value for $ {B}_j $ minimizing the JSD between the two distributions $ {E}_j\cdot {B}_j $ and $ {Q}_j^{Gas} $ . In the second step, we use the KS test statistic to find the value of $ {B}_j $ where both distributions are most similar. In both methods, we calculate the statistics over a predefined and sufficiently large discrete set of values for $ {B}_j $ , within which we assume the minimum value is located. If the minimum value is not included in this predefined discrete set, we would notice that the minimum value is either at the lower or upper boundary of the set, indicating that the range for exploration was chosen too narrowly.

We explicitly note that this minimization should be applied for every year $ j $ individually, as the SPF can differ in different years. Suppose we add an assumption about the mean reduced heat demand $ \gamma (h) $ over all buildings with a heat pump $ h\in {H}^{HP} $ , e.g., based on a heat cadaster. In this case, we can directly estimate the mean SPF over all existing heat pumps. We validate our approach by comparing our results to field trials for SPF computation.

3.1.1. Find B minimizing the distance using the JSD

The first approach to find the $ {B}_j^{\ast } $ where both distributions are most similar is to compute the JSD between $ {E}_j\cdot {B}_j $ and $ {Q}_j^{Gas} $ for a high resolved, discrete set of values of $ {B}_j $ , and take the $ {B}_j $ minimizing the JSD, which we denote as $ {B}_j^{\ast } $ . The JSD is a symmetric measure for the similarity of two distributions (see Lin (Reference Lin1991) for theoretical details)

(7) $$ \mathrm{JSD}\left(P \parallel R\right)=\frac{1}{2}{D}_{\mathrm{KL}}\left(P \parallel R\right)+\frac{1}{2}{D}_{\mathrm{KL}}\left(R \parallel P\right) $$

where $ P $ and $ R $ are two distributions and $ {D}_{\mathrm{KL}} $ denotes the Kullback–Leibler Divergence. It takes smaller values the more similar the distributions are and reaches 0 if and only if both distributions are equal. The upper bound of the JSD is given by $ {\log}_x2 $ with $ x $ being the basis of the logarithm used in the computation of the JSD (Lin, Reference Lin1991; Nielsen, Reference Nielsen2020). In this paper, we use $ x=e $ . To smooth the histogram of both distributions, we apply a kernel density estimation using an exponential kernel. We define the optimal value $ {B}_j^{\ast } $ for $ {B}_j $ based on the JSD as

(8) $$ {B}_j^{\ast } := \arg \underset{B_j}{\min}\hskip0.22em \mathrm{JSD}\left({E}_j\cdot {B}_j,{Q}_j^{Gas}\right). $$

3.1.2. Find B minimizing the KS test statistic

A disadvantage of finding $ B $ by minimizing the JSD between $ {E}_j\cdot {B}_j $ and $ {Q}_j^{Gas} $ is that this method does not provide a range of values for $ B $ where both distributions are similar in a statistically significant way. Therefore, we introduce another method for finding an optimal $ B $ based on a goodness-of-fit test. These tests can be divided into two groups, namely parametric and nonparametric tests. As we do not want to make an assumption about the underlying distributions of $ {E}_j\cdot {B}_j $ and $ {Q}_j^{Gas} $ , we employ an nonparametric goodness-of-fit test. According to Arnold and Emerson (Reference Arnold and Emerson2011), the most popular one in this group is the KS test. Compared to other nonparametric goodness-of-fit tests, the KS test is also robust to outliers (Arnold and Emerson, Reference Arnold and Emerson2011). In our case, we utilize the two-sample variant of the KS test. This means, that an empirical distribution, here $ {E}_j\cdot {B}_j $ , is compared against another empirical one, in this case $ {Q}_j^{Gas} $ . Thus, the tested null hypothesis has the form

(9) $$ {H}_0:{F}_{E_j\cdot {B}_j}={F}_{Q_j^{Gas}} $$

where $ {F}_X $ denotes the empirical cumulative distribution function of a variable $ X $ . The KS test statistic $ {d}_K $ is defined as

(10) $$ {d}_K\left({F}_{E_j\cdot {B}_j},{F}_{Q_j^{Gas}}\right)=\underset{t\in \mathrm{\mathbb{R}}}{\sup}\left\Vert {F}_{E_j\cdot {B}_j}(t)-{F}_{Q_j^{Gas}}(t)\right\Vert $$

following the literature (Lehmann and Romano, Reference Lehmann and Romano2022, Chapter 16.3). Consequently, we define the optimal value $ {B}_j^{\ast } $ for $ {B}_j $ based on the KS test as

(11) $$ {B}_j^{\ast } := \arg \underset{B_j}{\min }{d}_K\left({F}_{E_j\cdot {B}_j},{F}_{Q_j^{Gas}}\right). $$

The advantage of using the KS test as an instance of a statistical test compared to the JSD is that we get a range of values for $ {B}_j $ in which the two distributions overlap significantly at a given significance level. Therefore, we compute the KS test statistic $ {d}_K\left({F}_{E_j\cdot {B}_j},{F}_{Q_j^{Gas}}\right) $ from Equation 10 for every value of $ {B}_j $ in the predefined range of values to query and check, if the null hypothesis holds by checking

(12) $$ \frac{m\cdot n}{d} \cdot {d}_K\left({F}_{E_j\cdot {B}_j},{F}_{Q_j^{Gas}}\right)\ \le\ {j}_{\alpha } $$

where $ \alpha $ denotes the significance level, $ m $ and $ n $ denote the number of samples in $ {E}_j\cdot {B}_j $ and $ {Q}_j^{Gas} $ , $ d $ denotes the greatest common divisor for $ m $ and $ n $ , and $ {j}_{\alpha } $ denotes the critical value for rejecting the hypothesis given a significance level $ \alpha $ . A detailed discussion on the computation of the $ {j}_{\alpha } $ can be found for example in Comment 38 in Hollander et al. (Reference Hollander, Wolfe and Chicken2014) or in Hodges (Reference Hodges1958). The most important aspect from Equation (12) is that the distribution of $ {j}_{\alpha } $ is independent from the empirical distributions $ {E}_j\cdot {B}_j $ and $ {Q}_j^{Gas} $ , and the number of samples $ m $ and $ n $ .

6. Conclusion and outlook

The SPF is an important metric to estimate the annual electricity consumption of retrofitted heat pumps replacing existing gas furnaces. This work presents a novel method to estimate the SPF based on an unpaired dataset of energy consumption data of buildings within the same building class and located within the same city, where one group is formed by the buildings currently heated by heat pumps and the other group heated by gas furnaces. The SPF can be computed by minimizing either the JSD of these two unpaired distributions or the KS test statistic. We apply our method to an exemplary, recent dataset and validate the results with results from field trials. The resulting values using the KS test statistic for estimating the average SPF or $ B $ , i.e., the SPF divided by the insulation difference between the buildings with heat pump and those with gas furnace, are all statistically significant. The large differences in the results of the estimated values for SPF or $ B $ for the various years under consideration emphasize the initial assumption that different weather conditions have a massive impact on the performance of heat pumps. Additionally, our analysis shows that shifts in consumption patterns as caused by the European gas crisis in 2022 only affecting the heating patterns in gas-heating buildings reduce the interpretability of the resulting SPF estimates. Moreover, we estimate the annual electricity consumption of future retrofitted heat pumps that impact the local distribution system.

As discussed in the introduction, heat pumps will continuously replace gas furnaces in the future. Whereas it is effortful to get measurements of the SPF of existing heat pump installations in a city, this work helps researchers and utility companies to estimate the average SPF comparably easily based on a unpaired dataset of gas and heat pump electricity consumption that is fast to obtain by reading the smart meter values. Based on the SPF estimation, the additional electricity consumption caused by heat pumps can be estimated for recently passed years and projected into the future, assuming that the weather conditions seen in the years with data available will happen in the future again. Our results highlight that using one average estimate for the SPF will not adequately reflect the strong year-to-year variability in heat pump performance that our analysis has shown. Consequently, relying on an average year would fail to capture the full diversity of conditions, potentially leading to inaccurate energy consumption estimates and less reliable simulations. Thus, it is essential to consider the annual variations in SPF for more precise and comprehensive modeling.

Therefore, the data-driven estimation of the SPF and electricity consumption of retrofitted heat pumps can serve as a foundation for analyzing the potential carbon emission reductions when replacing gas furnaces with heat pumps. For example, such an analysis as presented in Bayer and Pruckner (Reference Bayer and Pruckner2024) is based on such a data-driven SPF estimation. Another technical question that remains open is how the average SPF will change with the ongoing development of smart grids. More sophisticated control strategies for heat pumps, such as those aiming to maximize photovoltaic (PV) self-consumption as presented by Yousefi et al. (Reference Yousefi, Hajizadeh, Soltani and Hredzak2021), or those trying to minimize the overall heat demand as showcased by Bayer and Pruckner (Reference Bayer and Pruckner2022), might change the SPF of an individual heat pump, as emphasized by Pospíšil et al. (Reference Pospšil, Špiláček and Charvát2019). Another aspect to consider is categorizing heat pumps based on their energy source, which can also affect the SPF. As we do not know the heat pump type for most of the obtained electricity profiles, we plan to develop a detection mechanism based on pattern recognition to identify the different heat pump types based on their hourly electricity consumption profile.