3.1 Econometric methodology identifying people’s beliefs

In this section, we present a summary of the panel-data techniques proposed by Gaglianone and Issler (Reference Gaglianone and Issler2021), which are used here to identify market expectations or people’s beliefs. These techniques are appropriate for forecasting a weakly stationary and ergodic univariate process { $y_{t}$ }.

In our context, the forecasts of inflation, $y_t$ , are taken from a survey of agents’ expectations and are computed conditional on information sets lagged $h$ periods. These $h$ -step-ahead forecasts of $y_t$ formed in period $t-h$ are labeled $f_{i,t}^{h}$ , for individual $i=1,\ldots,N$ . The index for time is $t=1,\ldots,T$ , and the index for horizons is $h=1,\ldots,H$ .

The forecast of each agent, $f_{i,t}^{h}$ , is obtained by minimizing a heterogeneous loss function, conditional on heterogeneous information as well. The individual information set has two components: common information, available to all agents, $i=1,\ldots,N$ , and idiosyncratic information available only to agent $i$ . Moreover, common information and idiosyncratic information are orthogonal to each other.

Agents do not know the conditional density function of inflation but can approximate it using a location-scale model, which is widely employed in the forecasting literature and encompasses several density functions used in practice.

Based on this framework, Gaglianone and Issler show that the optimal individual forecasts for agents $i=1,\ldots,N$ , $f_{i,t}^{h}$ , are related to the conditional expectation using only common information, $\mathbb{E}_{t-h}(y_{t})$ , by an affine function:

(2) \begin{equation} f_{i,t}^{h}=k_{i}^{h} + \beta _{i}^{h} \cdot{\mathbb{E}}_{t-h}(y_{t}) + \varepsilon _{i,t}^{h}, \end{equation}

where the error term, $\varepsilon _{i,t}^{h}$ , is related only to the idiosyncratic part of individual expectations and orthogonal to common information.

Based on this result, we chose to identify market expectations or people’s beliefs as the common component of individual expectations – ${\mathbb{E}}_{t-h}(y_{t})$ .

Gaglianone and Issler show how to estimate market expectations or people’s beliefs, ${\mathbb{E}}_{t-h}(y_{t})$ , under different assumptions for the sizes of $N$ and $T$ . Here, we will only cover the case of large $T$ and limited $N$ , since they argue that today’s surveys have exactly this setup, which fits perfectly the setup of the Focus Survey of the BCB. Take the cross-sectional average of (2) and use the well-known decomposition:

(3) \begin{equation} y_{t}={\mathbb{E}}_{t-h}(y_{t}) + \eta _{i,t}^{h}, \end{equation}

where $\eta _{i,t}^{h}$ is a martingale difference, to obtain the following moment conditions after using a set of instruments and the law of iterated expectations as:

(4) \begin{equation} \mathbb{E}[(\overline{f_{.,t}^{h}}-\overline{k^{h}}-\overline{\beta ^{h}} \cdot y_{t})\otimes z_{t-s}]=0, \end{equation}

where $\overline{f_{.,t}^{h}}=\frac{1}{N} \sum _{i=1}^{N}f_{i,t}^{h}$ , $\overline{k^{h}}=\frac{1}{N} \sum _{i=1}^{N}k_{i}^{h}$ , $\overline{\beta ^{h}}=\frac{1}{N} \sum _{i=1}^{N}\beta _{i}^{h}$ , and $z_{t-s}$ is a vector of valid instruments, $s\gt h$ , and $\otimes$ is the Kronecker product.

When $T \rightarrow \infty$ and $N$ is fixed, Gaglianone and Issler show that the feasible extended bias-corrected average forecast (BCAF), $\widehat{\mathbb{E}}_{t-h} (y_{t})$ , based on consistent generalized method of moment (GMM) estimates of the vector of parameters $\widehat{\overline{\theta ^{h}}}= \left [ \widehat{\overline{k^{h}}};\widehat{\overline{\beta ^{h}}}\right ]^{\prime}$ obeys:

(5) \begin{equation} {\mathbb{E}}_{t-h} (y_{t})= plim_{T\rightarrow \infty } \Bigg [\frac{1}{N}\sum _{i=1}^{N}\frac{f_{i,t}^{h}-\widehat{\overline{k^{h}}}}{\widehat{\overline{\beta ^{h}}}} \Bigg ]= plim_{T\rightarrow \infty } \widehat{\mathbb{E}}_{t-h} (y_{t}), \end{equation}

This result implies that $\mathbb{E}_{t-h}(y_{t})$ can be consistently estimated, as $T\rightarrow \infty$ , using:

(6) \begin{align} \widehat{\mathbb{E}}_{t-h}(y_{t})&=\frac{1}{N}\sum _{i=1}^{N}\frac{f_{i,t}^{h}-\widehat{\overline{k^{h}}}}{\widehat{\overline{\beta ^{h}}}}. \end{align}

As it is clear from equation (2), individual forecasts are in general not equal to the conditional expectation ${\mathbb{E}}_{t-h}(y_t)$ . This happens essentially because of asymmetric loss. Despite that, all forecasts depend on a common factor ${\mathbb{E}}_{t-h}(y_t)$ , which we can think of as a market forecast. Under a symmetric risk function for the market as a whole, it will be the optimal forecast for inflation, which justifies its label as the market expectation of inflation, or, as people’s beliefs, that is, Blinder (Reference Blinder2000). This is our main identification assumption here: we equate people’s beliefs or market expectations with ${\mathbb{E}}_{t-h}(y_t)$ . Its feasible version is estimated as in equation (6).

Next, we discuss how to obtain the asymptotic variance of $\widehat{\mathbb{E}}_{t-h}(y_{t})$ applying nonparametric methods.

3.2 HAC covariance matrix estimation

One of the original contributions of this paper is to derive the asymptotic variance of $\widehat{\mathbb{E}}_{t-h}(y_{t})$ . From equation (6), note that $\widehat{\mathbb{E}}_{t-h}(y_{t})$ is a function of the GMM estimates $\widehat{\overline{\theta ^{h}}}= \left [\widehat{\overline{k^{h}}};\widehat{\overline{\beta ^{h}}}\right ]^{\prime}$ . Hence, we will proceed here in two steps. Based on $T-$ asymptotics alone, c.f. equation (5), first we will employ a HAC covariance matrix estimation method to compute the asymptotic covariance matrix of these estimates. Second, we will use the Delta Method to compute the asymptotic variance of $\widehat{\mathbb{E}}_{t-h}(y_{t})$ .

HAC covariance matrix estimation has a wide application to moment conditions where their autocorrelation and heteroskedasticity properties are of unknown form. In this case, using nonparametric methods is the standard procedure. Newey and West (Reference Newey and West1987) and Andrews (Reference Andrews1991) are the main references.

Let $\Gamma _j$ denote the $j$ -th autocovariance of a stationary mean zero random vector $h_t(\theta )$ , for which a valid orthogonality condition of the form ${\mathbb{E}} [h_t(\theta )]=0$ applies, where $h_t(\theta )$ is a Kronecker product between a vector of regression errors $\epsilon _t(\theta )$ and a vector of instruments $z_{t-s}$ , that is, $h_t(\theta )=\epsilon _t(\theta ) \otimes z_{t-s}$ . Thus, $\Gamma _j={\mathbb{E}} [h_t(\theta ) h'_{\!\!t-j}(\theta )]$ . The long-run variance-covariance matrix of $h_t(\theta )$ is defined as the sum of all autocovariances. Since $\Gamma _j=\Gamma '_{-j}$ , we can write the long-run variance as:

(7) \begin{equation} S=\Gamma _0 + \sum _{j=1}^{\infty }(\Gamma _j+\Gamma '_{-j}), \end{equation}

as long as this sum converges.

As argued in White (Reference White1984), there are three cases of interest regarding this sum. The first is where $h_t(\theta )=\epsilon _t(\theta ) \otimes z_{t-s}$ is uncorrelated, so that $S=\Gamma _0$ . The second is where $h_t(\theta )=\epsilon _t(\theta ) \otimes z_{t-s}$ is finitely correlated, so that the sum can be truncated from $j=1$ to $j=m$ because covariances of order greater than $m$ are zero, so that $S=\Gamma _0 + \sum _{j=1}^{m}(\Gamma _j+\Gamma '_{\!\!-j})$ . The third and most interesting case is when $h_t(\theta )=\epsilon _t(\theta ) \otimes z_{t-s}$ is an asymptotically uncorrelated sequence, and an essential restriction being that $\Gamma _j \rightarrow 0$ as $j \rightarrow \infty$ . Therefore, we shall assume that $h_t(\theta )=\epsilon _t(\theta ) \otimes z_{t-s}$ is a mixing sequence, which suffices for asymptotic uncorrelatedness.

The idea of smoothing autocovariances in nonparametric covariance matrix estimation is to use a series of weights that obey certain properties to guarantee a positive semi-definite estimator for $S$ . Newey and West (Reference Newey and West1987) considered estimators defined as:

(8) \begin{equation} \widehat{S}=\widehat{\Gamma }_0 + \kappa (j,l) \sum _{j=1}^{l}(\widehat{\Gamma }_j+\widehat{\Gamma '}_{\!\!-j}), \end{equation}

where $\kappa (j,l)$ is a kernel weight that goes to zero as $j$ approaches $l$ , the bandwidth parameter. The idea is that covariances of higher order have less weight, and as they are estimated with less accuracy. Therefore, the use of a HAC estimator involves the specification of a kernel function and bandwidth parameter. In our main application, we use the Bartlett (Reference Bartlett1950) kernel as proposed by Newey and West (Reference Newey and West1987)Footnote ¹⁰ and the data-dependent method proposed by Newey and West (Reference Newey and West1994) to choose the bandwidth parameter.

Although Gaglianone and Issler (Reference Gaglianone and Issler2021) have analyzed two asymptotic cases, we will just focus here on the case when we let $T \rightarrow \infty$ and where $N$ is fixed, since it fits well the current surveys of expectations.

The population moment condition for the case where we let $T \rightarrow \infty$ and where $N$ is fixed is given by:

(9) \begin{equation} {\mathbb{E}} [h_t(\theta _{0}^{h})]=\mathbb{E}[(\overline{f_{.,t}^h}-\overline{k_{0}^{h}}-\overline{\beta _{0}^{h}} \cdot y_{t})\otimes z_{t-s}]=0, \end{equation}

where $\overline{\theta _{0}^{h}}=(\overline{k_{0}^{h}},\overline{\beta _{0}^{h}})'$ includes the true parameter values for each $h$ that solves equation (9). Under the suitable conditions assumed in Hansen (Reference Hansen1982), there exists a consistent estimator, $\widehat{\overline{\theta ^{h}}}$ , of the true parameter $\overline{\theta ^{h}_0}$ , such that ${\widehat{\overline{\theta ^{h}}}}\overset{p}{\rightarrow }$ $\overline{\theta ^{h}_0}$ , for all $h$ , where $z_{t-s}$ is a vector of instruments with $dim(z_{t-s})\gt 2$ .

To find the asymptotic distribution of $\widehat{\mathbb{E}}_{t-h}(y_{t})$ entails the following steps. First, as proved by Hansen (Reference Hansen1982), the efficient GMM estimator $\widehat{\overline{\theta ^{h}}}$ is asymptotically normal as follows:

(10) \begin{equation} \sqrt{T}(\widehat{\overline{\theta ^{h}}} - \overline{\theta _{0}^{h}}) \xrightarrow{d} \mathcal{N}\bigg (0,\bigg (G^h{{S}^{(h)}}^{-1}{G^h}'\bigg )^{-1}\bigg ), \end{equation}

where the optimal GMM weighting matrix is ${S^{(h)}}^{-1}$ , with $S^{(h)}=\mathbb{E}[h(\theta _{0}^{(h)})h(\theta _{0}^{(h)})']$ and where $G^h=plim_{T \rightarrow \infty } \frac{\partial h'({{\theta }^{h}})}{\partial{{\theta }^{h}}}\Big |_{\theta ^h=\widehat{{\overline{\theta ^{h}}}}}$ .

Second, from equation (6), note that $\widehat{\mathbb{E}}_{t-h}(y_{t})$ is a simple function of the GMM estimates $\widehat{\overline{\theta ^{h}}}= \left [\widehat{\overline{k^{h}}};\widehat{\overline{\beta ^{h}}}\right ]^{\prime}$ . Applying now the Delta Method, we can find a consistent estimate of the long-run variance of $\hat{E}_{t-h}(y_{t})$ as follows:

(11) \begin{equation} \widehat{V}^{(h)}=\left [\begin{array}{cc}\\ \frac{-1}{\widehat{\overline{\beta ^{h}}}^{^{^{\;}}}} & \frac{\frac{1}{N}\sum _{i=1}^{N}(-f_{i,t}^{h}+\widehat{\overline{{k^{h}}}})}{\widehat{({\overline{\beta ^{h}}})}^{2}}\end{array}\right ]{\bigg (\widehat{G^h}\widehat{{{S}^{(h)}}^{-1}}\widehat{{G^h}^{\prime }}\bigg )^{-1} \left [\begin{array}{c}-\frac{1}{{\widehat{\overline{\beta ^{h}}}}^{^{^{\;}}}} \\ \frac{ \frac{1}{N}\sum _{i=1}^{N} (-f_{i,t}^{h}+\widehat{\overline{k^{h}}})}{\widehat{({\overline{\beta ^{h}}})}^{2}}\end{array}\right ]}, \end{equation}

where the estimate of $G^h$ is given by $\frac{\partial h'({{\theta }^{h}})}{\partial{{\theta }^{h}}}\Big |_{\theta ^h=\widehat{{\overline{\theta ^{h}}}}}$ and the estimate of $S$ is given by:

(12) \begin{equation} \widehat{S}^{(h)}= \left [\widehat{\Gamma }_{0}(\widehat{{\overline{\theta ^{h}}}})+\sum _{j=1}^{l}\kappa (j,l)(\widehat{\Gamma }_{j}(\widehat{{\overline{\theta ^{h}}}})+\widehat{\Gamma }_{j}^{\prime }(\widehat{{\overline{\theta ^{h}}}}))\right ], \end{equation}

where,

(13) \begin{equation} \widehat{\Gamma }_{j}(\widehat{{\overline{\theta ^{h}}}})=\frac{1}{T}\sum _{t=j+1}^{T}h_{t}(\widehat{{\overline{\theta ^{h}}}})h_{t-j}^{\prime }(\widehat{{\overline{\theta ^{h}}}}). \end{equation}

Then, we obtain

(14) \begin{equation} \widehat{\mathbb{E}}_{t-h}(y_{t})) \overset{Asy}{\sim } \mathcal{N} \Big ({\mathbb{E}}_{t-h}(y_{t}),\frac{V^{(h)}}{T} \Big ), \end{equation}

where $V^{(h)}$ can be consistently estimated by $\widehat{V}^{(h)}$ from equation (11).

Based on this limiting distribution, we can construct asymptotic confidence intervals for $\widehat{\mathbb{E}}_{t-h}(y_{t})$ for each $h$ , which will be of great usefulness in the construction of our measure of credibility. We discuss that in the next section.

Once we have characterized the asymptotic distributions as in equations (11) and (14), we can construct 95% HAC robust confidence intervals to be compared with the explicit or tacit targets for inflation in each point in time. This will determine whether or not the central bank was credible in that period. As we all know, there is nothing magic about 95% confidence bands and perhaps a broader approach can be employed, considering different levels of credibility risk. The use of fan charts is an interesting approach, since it allows the assessment of uncertainties surrounding point forecasts. This technique gained momentum following the publication of fan charts by the Bank of England in 1996. Because we are using asymptotic results, we will not allow asymmetries as is common place when fan charts are employed. However, the technique of expressing risk in its different layers is an interesting extension of the current setup.

3.3 A credibility test and a new credibility index

Computing the asymptotic distribution of $\widehat{\mathbb{E}}_{t-h}(y_{t})$ for each $h$ , as discussed in the previous subsection, we can use this distribution to construct a new credibility index. The basic idea is that a centered version of $\widehat{\mathbb{E}}_{t-h}(y_{t})$ is asymptotically normally distributed with zero mean and covariance matrix ${V}^{(h)}$ , which can be consistently estimated by $\widehat{V}^{(h)}$ from equation (11). We will argue here that the area under the probability density function of the normal distribution between the inflation target and $\widehat{\mathbb{E}}_{t-h}(y_{t})$ can be used to construct an index of central bank credibility. As before, this index will be based on a sound statistical basis.

Based on $\widehat{\mathbb{E}}_{t-h}(y_{t}) \overset{Asy}{\sim } \mathcal{N} \Big ({\mathbb{E}}_{t-h}(y_{t}),\frac{V^{(h)}}{T}\Big )$ , we can establish the cumulative distribution function of $\widehat{\mathbb{E}}_{t-h}(y_{t})$ , which we label as $F(x)$ . Here, we use $F(x)$ to construct a credibility index ( $CI_{IS}$ ) that has the following properties: (i) if $\widehat{\mathbb{E}}_{t-h}(y_{t})=\pi ^{*}$ , where $\pi ^{*}$ is the central bank inflation target midpoint, $CI_{IS}$ attains its maximum value at $1$ and this would be the perfect credibility case; (ii) $CI_{IS}$ decreases as the distance between $\pi ^{*}$ and ${\mathbb{E}}_{t-h}(y_{t})=\pi ^{*}$ increases, asymptotically going to zero.

Figure 1 describes the idea behind the index that will be measured as the density of $F(x)$ between $\widehat{\mathbb{E}}_{t-h}(y_{t})$ and $\pi ^{*}$ , denoted in the graph by the blue area.

Figure 1. Normal distribution.

The new credibility index proposed here obeys

\begin{equation*} CI_{IS}= \begin {cases} 1-\frac {|F(\widehat {\mathbb {E}}_{t-h}(y_{t}))-F(\pi ^*)|}{1/2}; & \text {if } -\infty \lt \pi ^* \lt \infty. \end {cases} \end{equation*}

The proposed index has some advantages over the existing indices in the literature. First, it relies on a pure statistical criterion and it does not depend on ad hoc bounds, making its applicability much broader. Second, it is based on a structural model for survey expectations that yields a consistent estimate of the conditional expectation. Additionally, it also encompasses the case when the consensus forecast is free of bias.

Based on the distribution of $\widehat{\mathbb{E}}_{t-h}(y_{t})$ , and on the inflation target $\pi ^*_{t}$ , the definition of credibility is the following. A central bank is credible if:

(15) \begin{equation} \pi ^*_{t} \subset \left[\widehat{\mathbb{E}}_{t-h}(y_{t}) -1.96 {\left(\frac{\widehat{V}^{(h)}}{T}\right)}^{1/2}, \widehat{\mathbb{E}}_{t-h}(y_{t}) + 1.96 {\left(\frac{\widehat{V}^{(h)}}{T}\right)}^{1/2} \right ]. \end{equation}

In words, a central bank is credible if the confidence interval around $\widehat{\mathbb{E}}_{t-h}(y_{t})$ contains the official inflation target $\pi ^*_{t}$ . It is not credible otherwise. Therefore, our credibility definition is based on a statistical criterion, avoiding the use of ad hoc bounds.

3.4 Discussion

As stated from the outset, there is a consensus in the literature that Blinder (Reference Blinder2000) offers an uncontroversial definition of credibility when he states that “A central bank is credible if people believe it will do what it says.” Thus, we proceed in making this statement operational to measure and test the credibility of central banks.

The first issue discussed here relates to what people stand for in Blinder’s definition. While it is difficult to argue against the definition itself, it remains unclear what people refer to. One possibility points toward everyone in the economy, or to households who make atomistic decisions about consumption, labor supply (leisure), etc. Taken as a whole, their decisions ultimately deliver an aggregate inflation index that we observe and care about.

Unfortunately, we cannot observe everyone in the economy and even some of the best household surveys do not have information on consumer’s expectations on a regular basis. Moreover, households are small relative to the size of the economy, and their individual expectations will not have an impact on inflation, only their collective expectations.

A way to circumvent these problems is to use the beliefs of professional forecasters, extracting their common component, which is common in this literature. Because they are experts, not only their expectations are surveyed on a regular basis, but they also have the power to influence the market and the opinion of other agents, including the central bank, that is, they are not small. Indeed, most of them are active financial stakeholders, which further validates the methodology proposed here.Footnote ¹¹

Here, we equate people’s beliefs to the common component of individual expectations in the survey. This entails removing averages of the individual intercept bias and the individual slope bias, c.f. equation (6). One may argue against removing these biases since a forecast reflects a professional forecaster’s true belief. As noted by Gaglianone and Issler (Reference Gaglianone and Issler2021), individual biases are mainly due to the asymmetry of the loss function used by the forecaster and reflect the final use of an individual forecast. So, biases are admissible at the individual level. However, people’s beliefs are measured at the aggregate level and represent here market expectations. If we do not remove these biases, we will end up with a measure of market expectations that will be biased, which makes little sense.Footnote ¹²

A common way some economists have sought to define credibility and “anchoring” of expectations is to think about the cross-sectional distribution (standard deviation across forecasters) of their point expectations. When this distribution converges (to a point mass) on the inflation target, the central bank’s target is defined to be (fully) credible, see Reis (Reference Reis2021). One may wonder whether our panel-data approach imposes any restriction on the cross-sectional distribution of beliefs. Although we employ panel data to extract market expectations, since almost all current surveys have large $T$ and small $N$ , our asymptotic distributions relies exclusively on $T \rightarrow \infty$ . Hence, our approach matches our main statistics to the pattern of the surveys we find in practice, but we do not impose any restrictions on the cross-sectional distribution of beliefs.Footnote ¹³

Finally, although the BCB adopts a standard rule of having a fixed number as the inflation target, some countries employ ranges as targets, for example, 1%–3%, instead of 2% as a mean and bounds. If we assume a symmetric distribution, we can always find a midpoint target in this case, which can serve as the fixed target to which we compare beliefs. Note that the bounds supplied by the central banks themselves are not used in our proposed method, which avoids the incentive for them to inflate their bounds to always be on target. Indeed, the confidence intervals we compute come from the uncertainty of our estimate of market expectations. So, we let the “market” have the final word on uncertainty.Footnote ¹⁴

4.1 Data

We use the data available in the Focus Survey of forecasts of the BCB. This is a very rich database, which includes monthly and annual forecasts from roughly 250 institutions for every working day and for many important economic variables, such as different inflation indicators, exchange rates, GDP, industrial production, and balance of payments series. The survey collects data on professional forecasters, including banks, investment banks, other financial institutions, non-financial companies, consulting firms, academic institutions, etc.

The survey started in May 1999, initially collecting forecasts of around 50 institutions mainly on price indices and GDP growth. It quickly evolved to include around 250 institutions and survey data on a much broader basis. About 120 institutions are active in the system today. In November 2001, the online survey was created and in March 2010 it was improved, resulting in the present version of the Market Expectations SystemFootnote ¹⁵

Our focus is on the inflation rate measured by the Brazilian Broad Consumer Price Index (IPCA), because it is the official inflation target of the BCB. The Focus Survey contains short-term monthly forecasts, 12-month-ahead forecasts, and also year-end forecasts from 2 to 5 years ahead. These year-end forecasts are fixed-point forecasts, where the forecast horizon changes monthly as time evolves. In our main application, we will focus on 12-month ahead inflation expectations, since those are fixed-horizon forecasts that can be collected at the monthly frequency. Also, this horizon is large enough for inflation shocks to dissipate, since we do not want to associate the lack of credibility with the presence of mean-reverting shocks to inflation.

Our sample covers monthly inflation forecasts collected from November 2001 until April 2017. The survey is available since 1999, but data regarding expectations 12 months ahead are available only since November 2001. In each month $t$ , $t=1,\ldots,T$ , survey respondent $i$ , $i=1,\ldots,N$ , may inform her/his forecast for IPCA inflation rates for five calendar years, including the current year. In our sample, $T$ =208 months and $N$ is, on average, 60.

In recent years, the annual inflation rate in Brazil as measured by IPCA has increased considerably, reaching over 10% per year by the end of 2015. Inflation expectations for all horizons have also increased, as Figure 2 shows.

Figure 2. Focus fixed horizon inflation forecasts.

The data in Figure 2 are different across horizons, which deserves an explanation. There are fixed-horizon forecasts up to the 12-month horizon which can be extracted directly from the database in every month, posted by professional forecasters. For horizons from 24 months, 36 months, up to 48 months, there are no fixed-horizon forecasts posted in the database. What is shown in Figure 2 for these horizons are interpolated year-end forecasts using the method proposed by Dovern et al. (Reference Dovern, Fritsche and Slacalek2012).

There are two reasons why we focused only on 12-month ahead forecasts in testing for central-bank credibility: (i) For horizons from 24 through 48 months, “forecasts” are not actually posted on the database by agents but are constructed by interpolation. It is hard to imagine how different they are from proper fixed-horizon forecasts that would have been posted in the database; (ii) in Gaglianone et al. (Reference Gaglianone, Giacomini, Issler and Skreta2021), the authors have modeled inflation as an ARMA(1,1) process with monthly data. It has an estimated AR(1) coefficient of 0.44. This is the best ARMA model chosen by BIC. Its 12-month ahead impulse response is a multiplicative function of $(0.44)^{12}=0.000005$ , that is, it is virtually nil, showing that transitory shocks have very little impact on these forecasts. This is a necessary condition for choosing the forecast horizon for credibility tests, since short-run perturbations do not impact its final result. Choosing longer horizons would necessarily require the use of poor estimates for beliefs, which we avoid.

Figure 2 shows three distinct periods when we compare the consensus expectations 12-month ahead with the inflation target. The first is from 2002 until the end of 2003, where there is a sudden and strong deterioration of inflation expectations due to the uncertainty around the 2002 election, which was reversed sharply after it. The second period is from the end of 2003 until the end of 2010, being characterized by the stability of inflation expectation around the target midpoint. The third period starts in the beginning of 2011, where we observe a slow and sustained rise in inflation expectations. After President Rousseff’s impeachment (2016), there was a change in economic policy and in BCB. Soon after, expectations fell quickly toward the target midpoint.

Figure 3 compares the monthly consensus forecasts, $\overline{f_{.,t}^{h}}=\frac{1}{N} \sum _{i=1}^{N}f_{i,t}^{h}$ , for inflation with the actual inflation rate for the 12-month-ahead horizon. In this figure, the measure of consensus forecasts represented by the black line was forwarded 12 months to match inflation, which it is supposed to track. In each month, we can compare IPCA inflation over the last 12 months and its respective forecast made 12 months prior.

Figure 3. Forecasts vs actual inflation rate for the 12 months ahead horizon.

As is clear from Figure 3, the consensus forecast underestimates actual inflation quite often, and this is true in 68% of the sample.

4.2 Results

Table 1 presents the results of the GMMFootnote ¹⁶ estimation for the parameters $\overline{k^{h}}$ and $\overline{\beta ^{h}}$ , based on information about $f_{i,t}^{h}$ and $y_{t}$ . We use as instruments up to two lags of the consensuses forecasts ( $\overline{ f_{.,t}^{h}}$ ), up to two lags of the output gapFootnote ¹⁷ and up to four lags of the 12-month variation of the Commodity Price Index (IC-Br)Footnote ^{18 ,} Footnote ¹⁹ .

Table 1. GMM estimation results for the 12 months ahead forecast horizon

Notes: The set of instruments is up to two lags of the consensus forecasts, up to two lags of the output gap, and up to four lags of the 12-month variation of the Commodity Price Index (IC-Br).

* The sample used in estimation and in forecasting up to 12-month ahead.

Our sample starts in 2001 and out-of-sample forecasts are constructed for the period between 2007 and 2017Footnote ²⁰ , re-estimating the coefficients at the end of each year, and using them to construct true out-of-sample forecasts of inflation. In the first column of Table 1, the sample denotes the period used in estimation, which is used to perform out-of-sample forecasts up to 12 months ahead. For example, the sample from 2001 to 2006 generates out-of-sample forecasts for the year of 2007; the sample from 2001 to 2007 generates out-of sample forecasts for 2008, and so on, in a growing-window setup.

In Table 1, Hansen’s over-identifying restrictionFootnote ²¹ test is employed in order to check the validity of instruments and GMM estimates. For all sets of estimates, we are far from rejecting the null, which is evidence that the instrument set is appropriate and orthogonal to regression errors. In the last column, we test for joint significance of the coefficients [ $H_0$ : $\overline{k^{12}}=0$ and $\overline{\beta ^{12}}=1$ ]. If the null hypothesis is not rejected, the consensus forecast is equal to the Extended BCAF and there is no evidence of any bias in it. But, we reject the null hypothesis with great confidence in all cases, a strong evidence of biases for the consensus forecast, requiring their respective extraction to construct a consistent estimate for ${\mathbb{E}}_{t-h}(y_{t})$ .

Our estimates in Table 1 show that both mean intercept and mean slope parameters are statistically significant for all samples. The average intercept value is 2.26, ranging from 1.89 to 2.52 and the average slope value is 0.56, ranging from 0.47 to 0.63.

Figure 4 compares consensus forecasts with the estimated Extended BCAF, for the 12 months ahead horizon. The gray line represents the consensus forecasts for the 12 months ahead horizon, while the black line represents the estimated Extended BCAF for the same horizon. Notice that the behavior of the two is quite distinct, with our estimate of ${\mathbb{E}}_{t-h}(y_{t})$ below the consensus on early years and above it on later years of the sample.

Figure 4. Consensuses forecasts and extended BCAF (people’s beliefs).

The next step is to estimate robust (HAC) covariance matrices and standard deviations based on the asymptotic distribution of the estimated Extended BCAF and then construct the respective 95% confidence interval. Figure 5 includes $\widehat{\mathbb{E}}_{t-h}(y_{t})$ , its respective 95% robust confidence interval, and the inflation target from 2007:1 through 2017:4.

Figure 5. People’s beliefs (Extended BCAF) and confidence intervals.

In Figure 5, the 95% confidence interval contains the target (4.5% throughout) slightly above 65% of the time. Credibility is lost for the BCB in the period covering almost all the year of 2007. Here, beliefs were too low compared to the 4.5% target. Another period of no credibility is between mid-2013 and mid-2016. There, beliefs are too high compared to the target. As noted before, these were times when the increase in inflation expectations was widespread, indicating a de-anchoring of inflation expectations not consistent with the target of 4.5% annual inflation rate.

From mid-2016 onward, there was a very steep fall in inflation expectations. This happened at the same time when there was a change in the economic team of the Brazilian government, including the Central Bank administration and its board of directors. Perhaps these changes helped to consolidate the fall in inflation expectations, which were re-anchored according to our criterion.

Figure 6 shows the same result as Figure 5. However, comparisons between beliefs and targets are only made at the end of each year, since, technically, the Brazilian inflation-targeting program has only year-end targets. Results are virtually unchanged: there is no credibility in 2007 and from 2013 to 2016, with the BCB recovering credibility in 2017.

Figure 6. BCAF, confidence intervals, and year-end targets.

Figure 7 presents the credibility index proposed above. The shaded regions indicate when the target is outside the 95% confidence interval, therefore, when the central bank is not credible. From 2008 on, there is a decrease in the credibility index of the central bank, although with some volatility. This decrease is finally consolidated in 2013, with an almost complete loss of credibility according to our index. Agents’ forecasts were significantly above target, which shows a disbelief in the target and/or a disbelief in the capacity or the willingness of the central bank to bring current inflation toward the target. From August 2013 to July 2016, the credibility index fell close to zero and stayed around zero for three years, until august 2016, where there was a very steep rise in the index: it rose from 0.06 in July to 0.53 in September, reaching 1.0 in December. In the first months of 2017, the index stayed between 0.95 and 1.00, expectations falling below target. This is evidence of a strong recovery of central-bank credibility, and a strong re-anchoring of inflation expectations.

Figure 7. Credibility index.

Figure 8 compares our measure of central bank credibility with other indices proposed in the literature. Our index is labeled as the “IS Index.” The other indices are labeled as the “CK Index” (Cecchetti and Krause (Reference Cecchetti and Krause2002)), the “DGMS Index” (De Mendonça and de Guimarães e Souza (Reference De Mendonça and de Guimarães e Souza2009)), the “DM Index” (De Mendonça and de Guimarães e Souza (Reference De Mendonça and G. J.2007)), and the “LL Index” (Levieuge et al. (Reference Levieuge, Lucotte and Ringuedé2016)).

Figure 8. Comparison between credibility indices proposed in the literature.

4.3 Comparing our credibility index with other indices in the literature

First, the behavior of our index differs from all other indices for 2007 and the beginning of 2008: ours point out to a loss in credibility, since inflation expectations fell too much during this period—the Extended BCAF was below 2% for some months. All other credibility indices are above 0.4 and in some cases close to 1.0.

Second, our index shows a loss of credibility during the global crisis of 2008–2009, whereas the other indices do not. Again, this is due to inflation expectations being very low vis-a-vis the target. After the crisis, our index shows a prolonged period where the central bank keeps its credibility, which is confirmed by all other indices.

Finally, the IS, DM, and LL indices show credibility falling to close to zero between 2013 and 2016, but the CK and the DGMS indices are kept above 0.8. In our view, this is not consistent with a severe loss of credibility shown by a rise in expectations depicted in Figure 2. All three indices (IS, DM, and LL) show strong evidence of a rise in central bank credibility by the end of the sample, being either equal or very close to one.

One could argue that the DM and the LL indices are close to ours, especially because their dynamics are similar for most of the period 2007–2017. We disagree with that. As noted above, the DM index and the LL index are very different from ours from 2007 all the way to the beginning of 2008. While ours show no credibility for the BCB, theirs show no sign of that. Note that 2007 was a year where the BCB was not credible using our test, since the target was above market expectations and above the confidence interval.

Although the dynamic behavior of the DM index and the LL index is close to that of our index after 2008, we find that these indices are higher than ours, sometimes by a wide margin. For example, in mid-2015, the gap reached about 0.45 —our index was close to 0.05 (almost no credibility) and theirs was close to 0.50 (midpoint credibility). Smaller gaps are also observed from the end of 2010 through 2014.