1. Introduction
Disseminating economic data is a central mission of both the World Bank (WB) and the International Monetary Fund (IMF). The World Development Report, produced annually by the WB, has been the definitive repository for country-level aggregates since the 1970s, and the institution explicitly re-branded itself as a “Knowledge Bank” in the 1990s (Kramarz and Momani, Reference Kramarz and Momani2013). The IMF describes itself “as a global hub for knowledge on economic and financial issues,”Footnote 1 and its International Financial Statistics is the authoritative source for financial and macroeconomic data for 194 countries.
These financial institutions—together known as the Bretton Woods Institutions (BWIs)—also provide loans and grants to member states for development and stabilization purposes. This study suggests that the BWIs enhance the economic transparency of recipient countries through their lending activities. Governments that borrow from the BWIs provide more credible economic data of sufficient quality to be published in the WB's World Development Indicators.
BWI loans can encourage the disclosure of data through two mechanisms. First, BWI loans enhance a state's capacity for providing data, by freeing resources for improvements in a state's data infrastructure. BWI lending arrangements also typically offer technical expertise in data collection, aggregation, and dissemination. Second, BWI loans enhance a state's willingness to provide data—the policy conditions associated with the loans often include requirements to report key financial and economic data back to the lender.
Existing theory contends that transparency can destabilize autocracies, while transparency strengthens democratic regimes (Hollyer et al., Reference Hollyer, Rosendorff and Vreeland2015, Reference Hollyer, Rosendorff and Vreeland2019). Autocracies may therefore be disinclined to increase transparency of their own volition. If BWI lending increases transparency in autocracies, it is likely due to coercion—e.g. loan conditionality. By contrast, if democracies are only limited in providing economic data by state capacity, then BWI loans might help to increase economic transparency through the resources that they provide governments. Of course, there are domestic and international strategic considerations that may discourage governments from disclosing data—lest opponents at either level learn that the government is a better, or worse, performer than generally assumed. So it is not a priori clear that either regime would succumb to BWI encouragement or pressure to improve data reporting practices.
This paper offers rigorous empirical tests of the relationship between BWI loans and economic transparency. The HRV index (Hollyer et al., Reference Hollyer, Rosendorff and Vreeland2014), which treats economic transparency as a latent predictor of the reporting of data on 240 economic variables to the WB for publication in the World Development Indicators, is our dependent variable. The predictor is estimated using a Bayesian item response model, based on an underlying dataset covering 125 countries for the 1980–2010 period.
The HRV index is derived from the full spectrum of economic indicators that national governments report to international organizations. The largest movements in the index are driven by indicators with high “discrimination” scores: the current account balance, goods and services exports, and changes in foreign reserves. These also happen to be critical for BWI lending. Other items with high discrimination scores (leading to more variable rates of reporting) include private capital flows, foreign direct investment figures, and additional trade related data.Footnote 2
This index measures the presence or absence data, not the quality of the data per se. The World Development Indicators has explicit standards for data quality, so the index captures at least some minimal level of data quality. Governments may have strategic reasons to manipulate data however, and low quality data are sometimes accepted and misreported by international organizations (see, e.g., Kerner et al., Reference Kerner, Jerven and Beatty2017). HRV primarily reflects data reporting, and only crudely captures data quality—a more fine-grained measure would be necessary to capture more subtle forms of manipulation.
As explanatory variables, we consider loan commitments and, separately, their disbursements to governments from the two BWIs. Given existing work on transparency and regime type, we test for regime-type-specific effects of BWI commitments and disbursements, in addition to testing for pooled effects on all countries.
Empirically, there are two major sources of confounding that we address in the analysis: possible selection bias and carry-over effects. Because these organizations are devoted to promoting data disclosure, transparency itself may be an important factor that influences who receives BWI loans. For example, loans may be more likely to be assigned to governments that have high potential to improve their economic transparency. Related, carry-over effects are common concerns in panel data analysis with a reversible treatment. Recent previous participation in BWI programs may systematically change the incentive of a government to work with BWIs in a later program, which would violate the stable unit treatment value assumption (SUTVA) and lead to biased estimates of the effect of loans on transparency.
To identify the effects of BWI loans on transparency, we take advantage of recent developments in causal inference with panel data (Imai et al., Reference Imai, Kim and Wang2021). PanelMatch adjusts for carry-over effects by matching observations with an identical treatment history for a pre-specified time span. It corrects for selection bias by refining the matched sets by weighting or matching on covariates, and applies a difference-in-differences estimator to correct for a possible time trend and to estimate short- and long-term effects.
This matching approach to causal inference is not without limitations. The analysis achieves high internal validity—on the treated observations that have matches—at the expense of external validity, discarding the many observations that do not match. This turns out to be particularly serious for the WB data, which has a relatively small set of matched observations. Also, the method requires the treatment to be discrete and cannot investigate the effect of loan size as a continuous treatment.
We complement the PanelMatch approach with multilevel regression models, estimating the heterogeneous associations between loans and transparency across political regimes by fitting varying coefficients models. We include country-, year-, and regime-specific intercepts to control for unobserved confounding in multiple dimensions that cannot be matched on when using the PanelMatch approach.
We find that loan disbursements and commitments from the IMF and the WB are positively associated with economic transparency, and these associations are likely to be causal in democracies.
The scholarly community has become increasingly aware of the ways in which missing data plagues empirical research. Missing data also stands as an obstacle to policymaking as well as the work of nongovernmental organizations and activists. While the bulk of research on the BWIs focuses on the economic and political effects of their lending activities, disseminating data remains a core mission of these institutions. Our findings show that their lending activities, which are often critiqued for falling short of achieving economic goals, succeed in furthering the objective of promoting greater data availability—at least for democracies. As has been found in many other areas of research on BWIs, objectives are more readily achieved with they align with the political will of recipient governments. We conclude that BWI loans help democracies generate the public good of economic information.
These results are first steps toward understanding the broader relationship between international organizations and transparency. Further questions include whether BWI loans enhance other facets of transparency, including freedom of the press, the presence of freedom of information laws, and the disclosure of human rights practices. There are also broader questions to be addressed about the effects of other types of international organizations. Does development assistance from regional development banks enhance economic transparency? How does their effectiveness compare with that of China's new Asian Infrastructure Investment Bank (AIIB)? What are the effects of bilateral aid, and do they differ across democratic and autocratic creditor countries? This paper offers a state-of-the-art empirical framework to pursue the effects of alternative international institutions, assistance and other interventions on economic and other facets of transparency.
2. IOs and data collection and dissemination
2.1 The Bretton Woods Institutions
The WB started publishing the World Development Report in the 1970s, and it made knowledge-sharing one of its central purposes in the early 1980s. The WB has more publications and a higher citation rate per publication than the IMF, the OECD, the Brookings Institute, and the International Institute of Economics—with the IMF in second place (Kramarz and Momani, Reference Kramarz and Momani2013, 422).
The IMF similarly identifies information sharing as a central purpose. In the 1990s, the IMF, working in collaboration with the WB, established the General Data Dissemination System (GDDS), the Special Data Dissemination Standard (SDDS), and the Data Quality Assessment Framework (DQAF).Footnote 3 All members of these organizations have access to technical assistance through the GDDS or the SDDS.
While participation in these frameworks and standards is largely voluntary, BWI programs can require the collection of data relevant to the performance and creditworthiness of borrowing states. The IMF has leverage explicitly built into their loans through conditionality, whereby recipient governments are required to adopt specific policies in return for continued disbursements of the loans. Conditionality famously targets macroeconomic policies, but policy conditions can also explicitly include data requirements.Footnote 4 The IMF staff requires national statistics in order to monitor compliance, and to refine its policy advice based on local conditions. Loan-conditioned scrutiny of economic aggregates has led the IMF to collect previously missing or update erroneous economic data. In 2000, for example, the government of Pakistan admitted that the previous administration fudged tax data reported to the IMF; in a 2001 IMF arrangement, the government committed to “progress in tax administration reforms, and progress in monitoring fiscal expenditure” (Vreeland, Reference Vreeland2002). As part of their 2010 arrangement with Greece, the IMF required the “strengthening of public sector reporting mechanisms, including statistical aspects” after the country was condemned for falsifying data in the run up to the country's sovereign debt crisis in 2009 (Wyplosz and Sgherri, Reference Wyplosz and Sgherri2016, 33). While the IMF staff provides helpful guidance on data collection best practices, assisting governments with their capacity to collect data, policy conditionality gives the organization leverage to overcome a government's potential lack of willingness to share data publicly.
The WB does not rely as much on policy conditionality per se, but development project loans require the WB staff to approve only projects that are suitable for local conditions. Ultimate project approval is subject to an executive board decision, and the loan commitment stage can be held up indefinitely without proper economic justification—which requires data. Thus the WB enjoys leverage over governments at the commitment stage (although, lacking policy conditionality for many of their loans, they do not necessarily enjoy the same leverage as the IMF during the disbursement phase). So, similar to the IMF, the WB can have a good relationship with a government, providing friendly advice on best practices, but the WB can also play a coercive role, particularly at the commitment stage.
The BWIs require effective data collection and reporting in order to design, implement, and monitor the programs and projects that their loans are designed to support. The evidence of their effectiveness however, is mixed. Cooray and Vadlamannati (Reference Cooray and Vadlamannati2015) do not find any effect of participation in an IMF program on transparency. By contrast, Hollyer et al. (Reference Hollyer, Rosendorff and Vreeland2011) find a positive and statistically significant effect for a dichotomous indicator for IMF participation. These studies treat these indicators as control variables and neither focuses on empirical issues such as selection bias or carry-over effects. Neither study examines IMF loan commitments or disbursements (only program participation), nor do they consider the effects of WB lending.
2.2 Recipient states
All governments have a predisposition toward opacity. The glare of the media or the public eye may inhibit the implementation of preferred policies, or limit opportunities for rent seeking, patronage, and extortion. Too much information about government under-performance can activate political opponents in legislatures, or offer a focal point around which protest and mass political action can coordinate (Bueno de Mesquita and Downs, Reference Bueno de Mesquita and Downs2005). Data on (poor) economic performance may be beneficial to one's political opponents, and a public “bad” to government or regime supporters.
Data transparency has implications for a government's international relations too. If rival states recognize that a country is a better (or worse) performer than generally assumed, disclosure can generate strategic weaknesses. Indeed, it is not obvious that augmenting the informational environment is always optimal for international negotiations (Carnegie and Carson, Reference Carnegie and Carson2018, Reference Carnegie and Carson2019, Reference Carnegie and Carson2020). Data disclosure is not without controversy.
Transparency is also costly. Obtaining information about the economy may require effort—a free and inquiring media, the availability of freedom of information requests, an open system for collecting, aggregating and sharing data and knowledge: all require resources to establish.
Data transparency also has economic and political benefits. Better data improves resource allocation, reducing inefficiencies from imperfect information. Investment (both domestic and foreign fixed) increases with transparency (Hollyer et al., Reference Hollyer, Rosendorff and Vreeland2018, ch. 8), which improves economic performance and wages, especially in labor abundant states. Transparency also reduces the volatility of economic aggregates, all potentially enhancing the political survival of leaders (Mansfield and Reinhardt, Reference Mansfield and Reinhardt2008; Hollyer and Rosendorff, Reference Hollyer and Rosendorff2012; Shambaugh and Shen, Reference Shambaugh and Shen2018). Sometimes leaders use transparency to divert responsibility for poor economic outcomes—blaming external events rather than poor policy choices—and providing the data to prove it (Rosendorff, Reference Rosendorff2005).
Leaders balance the political and economic costs and benefits of transparency differently, resulting in varying levels of observed transparency. What might affect this balance, and shift the level of economic transparency a state provides? Firstly, improvements in the ability to collect, aggregate and disseminate information reduces the costs of transparency. Economic transparency has been found to rise with GDP per capita, suggesting that capacity constrains information flows (Hollyer et al., Reference Hollyer, Rosendorff and Vreeland2018, ch. 4). BWI loans might be able to help address capacity constraints. To the degree that a loan is conditioned on, and used for, improvements in capacity, it can lower the costs of data collection, and shift the balance toward providing more information about economic policies and outcomes.
The prospect of a BWI loan might also impact a leader's willingness to provide information. A leader may be willing to tolerate more transparency—risking coordination of political opponents, or exposure of corruption or maladministration—in return for BWI loans. The loans might be used to buy political support, reward cronies, fix investment and infrastructure, or pay down debt. When a loan is conditioned on enhanced transparency, a reluctant leader may concede to improved transparency in order to access “free” resources useful for enhancing survival in office.
Leaders are constrained by both state capacity and their own willingness to supply economic data to the public. BWI loans reduce the economic costs of informational collection and aggregation, and may enhance political prospects for leaders if the extra resources offset any potential risk of coordination by political rivals. We expect BWI loans to cause an increase in transparency.
2.3 Regime type
Many scholars contend that democracy is practically synonymous with transparency (Broz, Reference Broz2002). While democratic institutions may pressure governments to focus their efforts on highly visible policy areas over the obscure (Mani and Mukand, Reference Mani and Mukand2007) and these governments may actively seek to obfuscate certain policy areas (Kono, Reference Kono2006; Berliner and Erlich, Reference Berliner and Erlich2015), they still tend to disclose data at higher rates than autocracies.Footnote 5 In general, transparency helps stabilize democratic regimes, attract investment, and improve governance and reelection chances for leaders (Hollyer et al., Reference Hollyer, Rosendorff and Vreeland2018). Democracies, therefore, are generally more willing to be transparent—even when controlling for capacity.
Autocracies, by contrast, often resist sharing information publicly. Opacity makes it easier to reward political insiders for their support with draws from the national fiscus. Opacity permits leaders with fewer institutional constraints to engage in corruption or rent extraction, with little concern for angering rival elites or the polity at large (Boix and Svolik, Reference Boix and Svolik2013).
Autocrats can more easily manage the threat of unrest among the masses when individuals are ignorant of what other citizens think of their government's performance. Routinely withholding credible information about the economy may not stave off an individual's distaste for her government, but it does leave her in the dark as to what others think. When economic data is shared, it informs the higher-order beliefs of citizens, making it easier to coordinate on public displays of discontent with the ruling regime. Mass unrest becomes more likely with increased economic transparency (Hollyer et al., Reference Hollyer, Rosendorff and Vreeland2015).
We should therefore anticipate substantial differences in the tendency to disclose ex ante, before any BWI loan commitments or disbursements, across regime-types (Bueno de Mesquita et al., Reference Bueno de Mesquita, Smith, Siverson and Morrow2003; Hollyer et al., Reference Hollyer, Rosendorff and Vreeland2011). These differences may also impact the mechanisms linking BWI lending to governments’ tendency to disclose. For instance, the logic posited in the literature—that democracies have a stronger inherent incentive to disclose information than autocracies—implies that loan conditionality is more relevant for the latter than for the former. Autocrats will only disclose more information when coerced into doing so; democratic leaders may wish to increase disclosure even absent external pressures. This might argue for a larger effect of BWI lending—and particularly IMF lending, given its heavy use of conditionality—on transparency under autocratic rule. Conditionality induces autocrats to disclose where they otherwise would not. Democrats, by contrast, were already in compliance with such conditions, and so conditionality does little to alter their behavior (Downs et al., Reference Downs, Rocke and Barsoom1996).
In contrast, democracies might be impacted more by the financial resources and expertise that come with BWI programs. Democratic governments have a stronger incentive to make use of BWI resources to improve their states’ statistical practices. Autocrats, on the other hand, face little incentive to increase transparency beyond the bare minimum required as part of the conditions attached by the BWIs to their loans. This mechanism—one mediated by the effect of lending on state statistical capacity—might lead one to anticipate that BWI lending would have a larger effect on transparency under democratic rule.
No doubt other mechanisms may also give rise to discrepancies in the effect of BWI lending across regime-types. In our statistical analyses below, we fit models that allow for such heterogeneity. Our theoretical discussion does not provide a clear expectation for the direction of these effects; it does provide a strong basis to believe that such heterogeneity might arise. We treat any such discrepancies as offering suggestive evidence regarding which causal effects are most likely to link BWI lending to transparency scores: if loan conditionality and coercion dominate, the effect on transparency will be greater in autocracies; if capacity improvements dominate, the democratic states will be more responsive to BWI lending than their autocratic contemporaries.
3. Data
3.1 Measuring transparency
The HRV Transparency Index treats economic transparency as a latent predictor of states’ reporting of data on 240 economic variables to the WB for publication in the World Development Indicators (WDI). The variables include only measures that national governments collect; surveys and indicators constructed by other parties are excluded. The WB makes its data broadly available, and governments are unlikely to be able to hide this information from other audiences.
The index is estimated using a Bayesian item response model where z j,c,t ∈ {0, 1} denotes an indicator equal to 1 if country c reports WDI variable j in year t and equal to 0 otherwise:
where δ j is a parameter estimating how difficult the data on variable j are to collect (compared to other variables in the estimation) and β j is a discrimination parameter estimating how well patterns of missingness for item j predict patterns of missingness for other variables in the estimation. The term transparency c,t is the estimated index of a given country-year's propensity to disclose data.
The main dataset is available for 125 countries for the 1980–2010 period, for a maximum of 3,875 observations. In our effective sample, the dependent variable, transparency c,t, ranges from −10.870 to 9.981 with mean 1.178 and median 0.773; the distribution is right-skewed. As Figure 1(a) shows, the majority (61.755 percent) observations are greater than zero and more than 1000 negative observations are concentrated in the interval of ( − 1, 0). Figure 1(b) separates the observations by regime and plots stratified distributions. It shows that democracies (mean 2.54, sd 2.22) average higher transparency than autocracies (mean −0.52, SD 1.65), suggesting concerns about heterogeneity across regime-type. Our measure of regime type is the binary {0, 1} drawn from Cheibub et al. (Reference Cheibub, Gandhi and Vreeland2010).
Figure 1. Empirical Distributions of Transparency.
3.2 BWI loan commitments and disbursements
To measure the effect of BWI arrangements on economic transparency, we consider loan commitments and, separately, their disbursements to governments. We consider gross—not net—disbursements, and we test for pooled effects on all countries as well as regime-specific effects.
In our sample, governments receive, on average, about 165.26 and 184.77 million US dollars (USD) in loan commitments and disbursements annually, respectively, from the WB.Footnote 6 Governments receive, on average, 116.1 and 34.07 million Special Drawing Rights (SDR) as commitments and disbursements, respectively, from the IMF (about 166.65 and 48.90 million USD at current market rates) between 1980 and 2010.Footnote 7 The average amounts of IMF and WB loan commitments are similar, but sample-average IMF loan disbursements are much smaller than sample-average IMF loan commitments and also much smaller than sample-average WB disbursements. IMF loans are concentrated on a smaller set of country-years than the WB loans: Among the sample country-years, the proportion receiving loan commitments and disbursements from the WB is 0.537 and 0.588 respectively, compared to 0.141 and 0.098 from IMF. Among the 125 countries in the sample, 93 receive IMF loan commitments at some point, 84 receive IMF loan disbursements; for WB loans, 99 countries receive both loan commitments and disbursements.
Among those country-years under “treatment,” the average amount of loan commitments is 307.57 million USD and 821.28 million SDR (1,178.52 million USD) from the WB and the IMF respectively, and that of loan disbursements is 314.44 million USD and 348.38 million SDR (499.89 million USD). Figure 2 reports the distributions of how many years a country receives loan commitments or disbursements from the BWIs. Figure 3 shows the distributions of loan commitments and disbursements from the IMF and WB to democracies and autocracies.
Figure 2. Distributions of Loans across Time. Note: WB loans occur more frequently and last longer. The scales of the y-axis are different across the IMF/WB panels.
Figure 3. Loans and Regime Type. Note: Histograms of loan commitments and disbursements by the IMF and WB, stratified by regime type. The distributions are similar across the BWIs. See the supplementary material for the full treatment assignment plots.
3.3 Control variables
If confounders simultaneously affect the decision-making of loan commitments or disbursements and the level of transparency, we may have selection bias. Also, because the BWI's are committed to promoting economic transparency, the potential to improve transparency may be an important factor that they consider, suggesting an endogeneity problem. Analysis of our data reveals associations that justify these concerns. Table 1 shows the association between loan commitments/disbursements and the level of transparency, by regressing each of them (separately) on the HRV index. The BWIs appear to be more likely to provide loans to less transparent countries—especially less transparent democracies. Among autocracies, however, the evidence suggests that BWIs are more likely to provide loans to the governments that provide more publicly available data. These patterns hold for both BWIs, although more so for the WB.
Table 1. Possible selection problem: how BWI loan assignment is “affected” by transparency
Note: ***p < 0.001; **p < 0.01; *p < 0.05. Estimated coefficients of transparency on BWI loan commitments or disbursements. Less transparency is associated with a higher probability of receiving a loan (Dummy/Probit) as well as receiving larger loans (Size/Linear). The relationship appears in the democracy subsample, but not among autocracies.
To correct for potential selection bias, we control for variables that are likely to explain BWI loan distributions and are also associated with economic transparency. Those variables include the level of transparency in the previous year, GDP per capita, population and GDP, trade as percentage of GDP, the number of signed bilateral investment treaties, and whether a country is a WTO member. Political instability is also likely to be associated with both BWI loan decisions and transparency.Footnote 8 Developing countries are more likely to receive BWI loans during high-profile years in the international arena, so we control for an indicator of membership on the United Nations Security Council (UNSC). Region could be another predictor of economic transparency, and, at the same time, it is likely that the distribution of BWI loans is systematically different across regions. We further control for the geographical distance between the capital of a country and the headquarters of IMF and WB.Footnote 9 Finally, we correct for time-trends in the matching/causal inference analysis and unobserved heterogeneity across countries and regimes and over time in the regression analysis.
All control variables are lagged by one time period. A small portion of the data is missing, and these missing values are imputed, resulting in an effective sample size of 3,750 country-year observations.
4. Methods: panel matching and multilevel regressions
When dealing with panel data with reversible treatment, another source of endogeneity emerges from possible carry-over effects. A country that recently received a loan might be more (or less) likely to receive another. Treatment history matters, and the effect on transparency is not only determined by the current treatment status but may be also affected by the treatment assignment in previous years. Such carry-over effects violate the aforementioned SUTVA, and biases the estimated effects of interest.
4.1 A matching method for panel data
To address the problems of selection and carry-over effects, we apply the PanelMatch method (Imai et al., Reference Imai, Kim and Wang2021), which uses an extended difference-in-differences (DiD) design. PanelMatch adjusts for possible carry-over effects by matching observations with an identical treatment history for a pre-specified time span. Then it corrects for selection bias by refining the matched sets with covariate-weighting. Finally, it applies a DiD estimator to adjust for a possible time trend and to estimate short- and long-term effects.
This method estimates the following causal estimand:
where δ is the average treatment effect of the treated (ATT), and δ(F, L) indicates the ATT at the F th period after country i is financed at t, by adjusting for carry-over effects back to L periods before the treatment. X it is the binary treatment variable, and it is defined as whether country i is financed by a BWI at time t. The first term on the right-hand side in Equation (2) is the observed treated outcome, and the second term is the counterfactual. Here, country i needs to experience a change of treatment state from X i,t−1 = 0 to X i,t = 1.Footnote 10 The counterfactual outcome is defined as experiencing no such a change: X i,t−1 = 0 to X i,t = 0. The term, $\{ X_{i, t-l}\} _{l = 2}^L$
, is the treatment history back L periods.
The method corrects for the carryover effect for L periods by defining the counterfactual not only considering the treatment status at t − 1 and t, but requires the identical treatment history back to L periods of the factual and counterfactual. We choose a value of L to improve the credibility of the limited carryover effect assumption (Imai et al., Reference Imai, Kim and Wang2021, 7). But F is suggested to be chosen based mainly on substantive interest.
To estimate the causal effect, PanelMatch uses the following DiD estimator:
This estimator takes three steps: (1) for each treated observation indexed by it, we construct a matched set ${\cal M}_{it}$
of control units that share the identical treatment history from time t − L to t − 1; (2) then we use a set of covariates Z to construct the weights $w_{it}^{i'}$
for each of matched control units i′ in ${\cal M}_{it}$
to correct for selection bias; and (3) we apply the DiD estimator in Equation (3) to adjust for a time trend from t − 1 to t + F and estimate ATT for F = 0, 1, 2, ..., i.e., the instantaneous, short-run and long-run (cumulative) effects. The standard error of ATT is computed using a block bootstrap procedure. We utilize the R package PanelMatch 2.0 for the analysis. To calculate regime-specific ATTs, we specify the “moderator” argument as the democracy indicator.Footnote 11
This approach faces limitations. The analysis achieves high internal validity at the expense of external validity—features of our data lead PanelMatch to discard many treated observations. For the IMF analysis, the ratio of the number of matched treated observations to the total number of the treated observations is between 35 percent to 76 percent in the stratified and pooled analysis; for the WB the ratio is below 10 percent. Another shortcoming of the matching method is that it requires treatment to be discrete and cannot be used to investigate the effect of changes in the size of the loan. The average dose of IMF loans is very different from that of the WB (see Figure 3), and the size of loans size may be important in the degree of pressure the BWIs can apply. Moreover, the matching method is unable to correct for bias caused by unobserved attributes of countries.
4.2 Multilevel analysis
To investigate the external validity and generalizability of our results, we complement the causal inference with multilevel analysis. This approach also allows us to employ (approximately) continuous terms—i.e., the size of loan commitments and distributions—as explanatory variables. We use multilevel modeling to estimate the heterogeneous effects of BWI loans on transparency across political regimes by specifying varying-coefficient models, including the same controls as in the PanelMatch method. With multilevel modeling, we can control for unobserved heterogeneity with country-, year-, and regime-specific intercepts effects. The multilevel analysis stratifies by regime (as in the causal inference analysis), but uses all the information and treats the sub-samples as co-existing in a larger population. We consider both the intensive and extensive margins of loans—not just the presence of a loan but also its size. Information-based criteria, such as AIC, BIC, and log likelihood, can be used to decide whether the effects of BWI loans vary across regimes.
In addition, the effects estimated in PanelMatch only apply to the “treated” due to the limitation of our sample. In other words, we only know whether receiving loans affects transparency on the countries that actually receive the loans. But the relationship between loans and transparency estimated in the multilevel modeling is the average over both the “treated” and the “control” observations.
We use the following specification to conduct multilevel analysis:
Equation (4) is the individual-level regression: i indicates a country, and t indicates a year. The subscript of parameters α j[it] and β j[it] denotes whether the observation of country i at time t is a democracy (j[it]=1)or autocracy (j[it]=0). That is, α j[it] = α 1/α 0 and β j[it] = β 1/β 0 are regime-specific intercepts and coefficients.
Equations (5) and (6) are the specifications of varying intercepts and regime-specific coefficient, reflecting heterogeneity in multiple dimensions of time, unit, and regime type. As shown in Equation (5), β j consists of a shared part, β, by both regimes and a varying part, $\epsilon _j \sim {\cal N} ( 0,\; \, \sigma _{2j}^2)$
, that is different across regimes. As opposed to a subsample analysis, the multilevel model admits the fact that different regimes coexist in a larger population and may share some common attributes that lead to a shared loan-to-transparency mechanism, β. The heterogeneous effect across regimes is captured by the regime-specific random effect $\epsilon _j$
. The regime-specific intercept, α j, captures the impact of regime heterogeneity on the outcome. We can see that, in multilevel modeling, regime type is treated as a macro-institutional context that shapes and moderates the mechanism through which loans affect transparency.
There are several advantages of applying multilevel modeling to test our hypotheses. First, compared with fixed effects models, the multilevel model can include more than two-way fixed effects. Its model specification and estimation strategy allow it to have unit-invariant or time-invariant covariates at the same time as fixed effects. Second, multilevel modeling is more efficient than split-sample (democracy or autocracy) regressions, because it can borrow information across groups (Gelman and Hill, Reference Gelman and Hill2006). Third, as opposed to regressions with an interactive term (i.e., regime*loan.IO), multilevel modeling treats regime type as part of the macro-level institutional environment and the conditioning goes from the macro context to micro behavior (loan.IO|regime), whereas in an interactive term, the components have exchangeable positions and the conditioning goes both ways, β loan.IO|regime and β regime|loan.IO, but the latter is difficult to interpret.Footnote 12
We conduct multilevel analyses by estimating and comparing three different specifications, including models with an invariant β loan.IO across regimes (M1), models with regime-specific coefficients β loan.IO|regime (M2), and models with an interactive term regime*loan.IO (M3). The results are largely robust to alternative measures of the key variables and various methods.
5. Results
5.1 Matching
We adjust carry-over effects for three years (L = 3) in the treatment history. The choice of the value of L is arbitrary, and we have to balance the trade-off between the size of matched set and the length of matched treatment history: the size of matched set decreases with a larger L, and an L larger than 3 leads to much smaller size of matched set based on our sample. We set F = 8 to take a relatively long time horizon, since the size of matched set is not too sensitive to the choice of F. Only those observations that change treatment status less than F years before the end of the sample period are excluded.Footnote 13 We use the Covariate Balancing Propensity method to correct for selection bias and improve balancing; alternative methods such as propensity score weighting and Mahalanobis matching generate similar results.
Figures 4 and 5 report the estimates and 95 percent confidence intervals of ATTs of loan commitments and disbursements from t 0 to t 8, where the effect at t 0 is the instantaneous effect. We also report the number of matched countries and matched observations in the figures. The first column displays the results when pooling all countries; the second and third columns are the ATTs of democracies and autocracies, respectively.
Figure 4. IMF Loans: Causal Effects (L = 3, F = 8).
Figure 5. WB Loans: Causal Effects (L = 3, F = 8).
The effects of IMF loan commitments (top panel) and disbursements (bottom panel) have similar patterns (Figure 4). Column (1) suggests that IMF loan commitments and disbursements both significantly increase the level of transparency of countries that participate in loan programs. The significant positive effect lasts for at least eight years after receiving loan commitments and increases over time, whereas the effects of IMF loan disbursements reach conventional thresholds of significance only in the short-term. These estimates suggest that pressure to provide more and better data is most acute at the time of the loan, but the influence of the IMF over data collection becomes more varied and uncertain (not statistically significant) after the initial loan disbursement. The magnitude of the point estimate of the long-term (t+8) effect in the top panel of column 1 (≈ 0.3 on the HRV index scale) is approximately equal to ${1\over 8}$
of a standard deviation in index values.
Columns (2) and (3) report the ATTs of IMF loans on transparency across regimes. The figures distinctly indicate that the effects are exclusively observed in democracies. Both loan commitments and disbursements significantly enhance transparency in democracies, with the effects being both enduring and increasing. However, it appears that IMF loans do not have any reliable effect on improvements in transparency in autocracies.
Turning to the WB, we note that the treatment assignment patterns of its loans are significantly different from those of IMF: the WB tends to continuously finance countries over a longer time horizon. Figure 5 estimates the effect of WB loan commitments (top panel) and disbursements (bottom panel) on transparency up to eight years after the loan. Loan commitments from the WB have a long-term effect on the level of transparency of recipient countries. The effect develops gradually and becomes statistically significant in the fifth year of the loan. However, there is no discernible significant effect of WB loan disbursements, and the point estimates appear to trend negatively with the passage of time.
When examining the distinct effects of the WB on democracies and autocracies, it becomes evident that the effect is exclusive to democracies, mirroring the pattern seen with IMF loans. In democracies, there is a long-term effect of WB loan commitments and a short-term effect of WB loan disbursements. However, in non-democracies, WB loan commitments show no significant effect. Surprisingly, economic transparency in non-democracies even appears to decrease after five years of receiving WB loans.
5.2 Multilevel regression analysis
We complement the matching approach with regression analysis. We do not interpret the regression coefficients here as “causal effects” because the two-way fixed effects linear regression and its variants (such as the multilevel analysis that we use) are only equivalent to the difference-in-differences estimator “under the simplest setting with two groups and two time periods.”Footnote 14 We can, however, test the association between loan amounts and the level of transparency with the whole sample rather than a small matched subset, while controlling for covariates.
All models include year-specific, country-specific, and regime-specific intercepts. For the independent variables, we use both loan size and a binary loan indicator (for commitments and disbursements, respectively). The information-based criteria for model selection (AIC, BIC, and log likelihood) all indicate a preference for the models with a constant coefficient across regimes (M1). That is, the regression analysis suggests that BWI loans do not impact the data disclosure of democratic governments differently from how they affect autocracies.
Figure 6 reports the estimates and 95 percent confidence intervals of the association between BWI loan amounts and transparency based on the three different specifications. Different model specifications generally agree with each other, and suggest a positive association, broadly with the previous analysis, although not without some nuances.
Figure 6. Estimated Coefficients of BWI Loans on Transparency, Note: Coefficients from the multilevel models. The left panel considers the size of a loan (at the logarithmic scale) as the independent variable; the right panel uses a dichotomous indicator for a country-year with or without a loan.
Interestingly, the estimate profiles are very similar when coding the explanatory variables as discrete; they differ only in magnitude and turning “on” or “off” the treatment is associated with an expected change of transparency by about about 0.06 to 0.07 units on the HRV scale for WB and 0.04 to 0.06 points for IMF loans. Given that the preponderance of the variance in loan size is between country-years that receive a commitment/disbursement and those that do not (and so are coded as zero), we focus our discussion on the binary controls. The linearity assumption underlying the (quasi-)continuous treatment may be strong in this instance.
The point estimates on the binary indicators in Figure 6(b) are substantively small. A movement of 0.06 units on the HRV index scale is comparable to the annual change in transparency scores witnessed in Vietnam during the late-1980s and early-1990s, following the reforms of the 1986 Party Congress. However, recall that our specifications include a lag of transparency (LDV) in the estimating equation, implying a dynamic relationship in which the marginal effect of a loan evolves geometrically over time (Beck and Katz, Reference Beck and Katz2011). The coefficient on the LDV, as reported in Table 2 is substantively large (approximately 0.97 in all specifications). Were the loan dummy to switch from a value of zero to that of one permanently, this implies a long-run equilibrium effect roughly 30 times the size of the point estimate, a 1.8 point shift in transparency scores, or 0.78 standard deviations.
Table 2. M1: loan commitment/Disbursement size and transparency
Note: ***p < 0.001; **p < 0.01; *p < 0.05. Multilevel regression for pooled sample.
BWI lending (particularly IMF lending) often doesn't follow this pattern (see the discussion of the data structure in the supplementary material), and so such an interpretation involves extrapolation from the model beyond the coverage of the data and should be taken with a grain of salt. However, the dynamics of the model do imply that changes in transparency associated with BWI lending are persistent and grow over time, as is also consistent with the pattern PanelMatch discovers for the ATT in democracies.Footnote 15 Vietnam witnessed such a pattern of consistent liberalization over the decade following the 1986 Party Congress and—while the increase in transparency year-to-year was small, the cumulative rise reflected one of the more pronounced examples of autocratic liberalization according to the HRV index.
Among the regression models, the estimates based on M1 (the pooled effects) are roughly the average of the regime-specific effects based on models with regime-varying coefficients (M2.auto and M2.demo) or on models with an interactive term (M3.auto and M3.demo). M3 suggests bigger regime-differences than M2 does, but the 95 percent confidence intervals of regime-specific estimates are largely overlapping in both models, confirming what is suggested by model comparison with information-based criteria—the association is not different across regimes. Furthermore, M3 finds neither IMF loan commitments nor IMF disbursements have a significant association with the economic transparency of democracies, either in terms their size or dichotomous intervention. M1 and M2 suggest that pooled and regime-specific effects are all statistically significant, whereas M2, the model with an interactive term suggests neither IMF disbursements nor IMF commitments have a significant effect on democracies.
Table 2 reports the estimated coefficients associated with the control variables based on the best model with a pooled effect of BWI loans, using the size of BWI loans (M1 in Figure 6(a)).Footnote 16
As noted above, we find strong autocorrelation in the outcome variable, the HRV index, which is greater than 0.95 after we control for time-varying covariates.Footnote 17 The lagged transparency level explains most of the variation in the dependent variables, with R 2 = 0.970 in the simple regression on transparency level with the lagged level. The lagged dependent variable may be absorbing the effects of weaker factors, which is the reason that most of our control variables do not have a statistically significant associations with transparency. In the models with an invariant coefficient on loans across regime types, we are able to include the variable Democracy as a control variable. As expected, being democratic is positively associated with transparency.Footnote 18 There are two region-indicators (Asia and Europe) that are significantly associated with a higher-level of transparency, compared to the baseline category of Africa.
Substantively, the multilevel analysis produces some different findings compared to the results from the matching/causal inference approach. While the causal analysis suggests that the effect is, in general, stronger on democracies than on autocracies, the multilevel analysis finds no significant difference across regimes. Moreover, the multilevel analysis finds that the association between WB loans and transparency is significant for both disbursements and commitments, whereas the matching/causal inference does not estimate much effect of WB disbursements. Also, regressions weakly suggest that IMF disbursements and commitments may not be significantly associated with an increase of economic transparency in democracies, whereas PanelMatch finds both long-run and short-run effects of IMF loans on democracies. The consistent finding across the methods is a positive and statistically significant association between BWI loans and the improvement of the level of economic transparency in developing countries.
6. Conclusion
The statistics made available by the IMF and the WB shape much of the research and policy discussions among international affairs professionals. We show that loans from these institutions increase the amount of available economic data provided by recipient governments.
Our results are corroborated by two methods—the PanelMatch approach to causal inference of Imai et al. (Reference Imai, Kim and Wang2021) and multilevel regression analysis. PanelMatch enables us to estimate causal effects with a high degree of internal validity, but potentially at the expense of external validity due to a lack of matching observations. The complimentary multilevel modeling approach does not estimate causal effects, but enables us to estimate heterogeneous associations of loans across political regimes with a varying coefficients and controls for unobserved heterogeneity with country-, year-, and regime-varying intercepts.
Both approaches confirm a positive association between BWI loans and increased transparency. PanelMatch suggests that the effect for democracies is causal. The results suggest that the effect comes through capacity building—the loans and technical advice—rather than through coerced policy conditionality. While our evidence of mechanisms is speculative, this pattern is consistent with the more robust positive relationship between BWI lending and transparency in democracies.
Other forms of development assistance—from other multilateral organizations and from bilateral donor countries—may also improve rates of data disclosure by democratic governments. We argue that BWI loans help democracies overcome capacity constraints, but BWIs can help governments in this regard through two potential mechanisms: funding and technical advice. If democracies only require funding, then a wide range of development assistance may improve economic transparency under democracy. But if the key to success is technical advice, then BWI assistance may outshine other providers. This question may become especially important as more developing countries consider borrowing from new organizations, such as the AIIB (Qian et al., Reference Qian, Vreeland and Zhao2023).
Alternative facets of transparency under democracy may also be enhanced by international organizations—for example, certain human rights organizations require states to self-certify rights compliance. Future research might examine a broader range of international organizations and measures of transparency.
The results for autocratic regimes were inconclusive; further research into the varieties of autocracy or other institutional features may yield insights. As a preliminary step in this direction, we include estimates of the ATT varying across six definitions of regime-type {parliamentary democracy, presidential democracy, semi-presidential democracy, military autocracy, civilian autocracy, monarchy} in the supplementary material. These produce some evidence that BWI lending has a negative ATT in military dictatorships, though these results should be treated as speculative given the sample size.
While the World Bank, IMF, and other international organizations have implemented reporting practices that discourage the publication of low-quality data, regular updates and corrections to pre-existing data suggest that data quality varies systematically across countries with different levels of economic development and different political institutions. Future research might consider whether more frequent borrowing from the BWIs results in governments producing data that are less likely to require corrections over time.
The lending activities of the BWIs improve the production and dissemination of economic information by the democratic governments that borrow from them. BWI lending has enhanced the collection and dissemination of data, a vital resource for investors, policymakers, and scholars, who use them to make better decisions for economic development.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/psrm.2024.28. To obtain replication material for this article, https://doi.org/10.7910/DVN/SYFQLQ
Acknowledgements
We thank Bernhard Reinsberg and participants at the Foreign Aid & Political Gains Workshop at the University of Haifa in 2023, Tianyang Xi and participants at the National Instutute of Development Workshop at Peking University in 2022, Yaqing Qin and participants at the Global Governance and International Relations Workshop at Shandong University in 2022, and the editors and anonymous reviewers at Political Science Research and Methods for their very helpful suggestions. The authors contributed equally and are listed in alphabetical order. James Hollyer thanks the Lippincott foundation for research support.
Conflict of interest
None.
Open access
