Fostering ownership

There have been at least three notable examples of performance failure in efforts to implement the ownership strategy: failures in efforts to reduce the number of structural conditions, to build meaningful participation into the PRSP process, and to operationalize the PRSP. The Independent Evaluation Office (IEO) at the International Monetary Fund (IMF) released a report in November of 2007 on the IMF’s streamlining initiative that concluded that although conditions were now more focused on “core” IMF areas, the streamlining initiative had not actually reduced the number of structural conditions.⁴ Moreover, Fund programs still included conditions that were not deemed critical to program objectives.⁵ The streamlining exercise thus largely failed to reign in structural conditionality.

In the case of the PRSP, both external and internal reports suggest that the program has failed to genuinely engage civil society in preparing the crucial document. A number of studies of the Bolivian experience with the PRSP, generally seen by the IFIs as one of their successes, have pointed to the narrow scope of participation. As Nadia Molenars and Robrecht Renard put it, the selection of civil society participants was politicized, as it was dominated by local government officials, at the same time as the agenda for debate was depoliticized – focused almost exclusively on social rather than political and economic issues.⁶ Although the context and the mechanisms of participation were different in Uganda, Laure-Hélène Piron and Andy Norton raise similar concerns about the fact that certain groups were included while others (deemed more “political,” like unions) were excluded from the process, and that economic and, above all, political questions were kept off the table.⁷

The IMF and World Bank also undertook two major joint reviews of the PRSP process, while the IMF’s IEO evaluated the IMF’s role in PRSPs.⁸ Not surprisingly, the internal reviews have generally been positive about the overall direction of the programs. Yet despite their optimism, the internal reviews also highlighted some significant concerns about the programs, including limited participation, insufficient policy space, donor-driven processes and inadequate tailoring. While later generations of PRSPs have responded at least in part to such criticisms, the early enthusiasm about the policy both inside and outside the IFIs has faded.

Finally, the PRSP has significant technical limitations. The ingenuity of the PRSP rests in part on its capacity to translate a broad range of concerns into a single inscription designed to coordinate development activity. Yet it is not clear whether the PRSP is a useful inscription at all. As one former senior Bank staff member put it:

The effectiveness of the PRSP depends on the translation of a multitude of voices into a single coherent text, and the enrolment of a much wider range of actors in a common project. Yet in those cases in which a genuine attempt is made to bring a diverse range of people into the conversation, their very multiplicity resists straightforward translation into a tidy bureaucratic document, reducing the PRSP’s usefulness as a governance tool.

These practical failures have helped to erode key actors’ support for the ownership strategy, and undermined efforts to black-box the strategy. Many of the IFI staff I spoke with expressed frustration with the lack of prioritization in PRSPs and the considerable difficulties of using them as guides for the various lending policies and programs they were supposed to inform.¹⁰ Despite efforts to resolve these tensions and “mind the gaps” between the PRSP and government and donor policies, problems remain.¹¹ Over the several years that I undertook these interviews, it was clear that there was less attention and energy being put into the PRSPs, partly because they simply have not been easy to put into practice. Many staff members feel outside of their comfort zone when asked to assess ownership, or to consider the impact of a policy on poverty.¹² Inside the IMF, in particular, there has been considerable ambivalence about the expansion of the organization’s mandate.¹³ Staff at both organizations admit that the PRSP is no longer as important as it once was, and note that ownership has become a less important lens for designing new policies.

Developing global standards

The universal claims of the strategy of standardization depend on its capacity to sustain a consensus on what counts as a global standard. Yet that consensus – insofar as it was ever achieved – is unravelling. The strategy of standardization has always been a compromise of sorts – an attempt to develop soft universals that fall between a fully contextualist, case-by-case approach, and hard economic rules of the kind that characterized the earlier structural adjustment era. At the same time, these standards are far more ambitious in their scope than earlier economic principles, moving into highly contestable social and political terrain. As a result, these universals are far more obviously subjective and constructed than those economic principles that underpinned the structural adjustment era: they are therefore harder to black-box, leading to ongoing ambivalence about the strategy among the IFIs.

There has always been some disagreement among institutional actors about whether these new standards are truly universal. Even as the idea of “best practices” was gaining ground at the IMF in 1997, some Executive Board members expressed reservations about their appropriateness.¹⁴ The World Bank’s 2000 and 2002 reports on governance also questioned the appropriateness of the “best practices” approach that had dominated discussions to that point.¹⁵ With the arrival of Paul Wolfowitz as President in 2005, and his increased emphasis on corruption as the centrepiece of the good governance agenda, the universality of the strategy again became a subject of heated debate among Board members as they challenged the appropriateness of translating good governance performance into a set of standardized indicators.¹⁶

Whereas these challenges have come from those opposed to excessive standardizing, there are also those who would like to return to something closer to universal rules. At the IMF, there have been persistent calls for the institution to withdraw from more overtly political issues like governance altogether. It was partly because of such concerns that the Board made the decision to streamline conditionality. Since that time, there have been growing calls from powerful members, particularly the United States, for the Fund to narrow its activities to those areas clearly within its mandate.¹⁷ There have also always been those, like the British government, who have argued that standards should be mandatory, an argument that re-emerged in the wake of the recent global financial crisis.¹⁸

Although the idea of global standards has certainly become pervasive and generally accepted in development financing circles, these examples make it clear that the strategy has only ever been partly black-boxed. As Lampland and Star suggest, the seeming neutrality of standards “obscures the enormous amount of work needed to stabilize knowledge, freeze action, delete outliers and residuals, and facilitate use.”¹⁹ In the case of the strategy to develop global standards, this work of stabilizing and purifying is ongoing and continues to be contested.

In addition to these internal debates about the standardization strategy, there is evidence to suggest that these new techniques have not had the productive and performative effects that they were intended to have. As I noted above, the publication of inscriptions like the ROSC or scorecards on government actions is supposed have the performative effect of empowering citizens and market participants to demand government reforms. It is not at all clear, however, that market actors are behaving according to plan. The IMF and World Bank’s own evaluations of the standards and codes initiative found that market participants were not using the data on government compliance published in the ROSCs.²⁰ Moreover, although the IMF has regularly cited studies that suggest that markets have rewarded economies that are more transparent, even they admit that the evidence remains inconclusive at best.²¹ Despite these setbacks, the IMF has continued with the standards and codes initiative while scaling back expectations that the publication of the ROSCs will mobilize market actors to reward or punish participating governments.²²

Managing risk and vulnerability

Attempts to measure and manage risk and vulnerability, in their turn, face methodological difficulties that threaten the fragile expert consensus around the strategy. Fund and World Bank staff have found that a country’s or individual’s vulnerability to shocks depends in considerable measure on political and social factors that resist easy calculation. While the many acronymic indices produced as part of risk and vulnerability assessments – CPIA, PEFA, DSA and RVA – help to black-box the assumptions required to quantify these complex factors, they nonetheless remain vulnerable to criticism.²³

As I noted in Chapter 7, a number of senior economists at the World Bank were initially critical of the social risk framework. While this resistance has faded as social protection has become integrated into the institution, there is still considerable scepticism about this approach to measuring poverty.²⁴ Moreover, although staff have been at pains to emphasize the innovative character of the strategies for measuring vulnerability and risk, they continue to encounter difficulties in obtaining the right kind of data given the poor statistical capacities of many of the countries that they are dealing with.²⁵ Debates continue not only about the meaning of different measures but also on their ability to translate into new practices.²⁶ In fact, after almost a decade of RVAs, Social Protection staff concluded that they had not taken off and they were therefore dropped as institutional practices.²⁷

The IMF’s debt sustainability and vulnerability assessments, in their turn, have also faced methodological limits. Staff, Board members and NGOs have expressed doubts about these metrics, challenging the heavy reliance on the CPIA as a proxy for institutional capacity. Although Fund staff have been careful to emphasize the objectivity of the initial stage of the evaluation process, they remain sensitive to the fact that, as one document put it, any evaluation of vulnerability “would depend heavily on assessments of macroeconomic and public financial management capacity, which would inevitably have a subjective element.”²⁸

Academic critics have also raised concerns about the accuracy of the Fund’s DSA. As Charles Wyplosz notes in an analysis of the problem: “while price stability or full employment can both be measured with a reasonable degree of precision, debt sustainability cannot even be measured directly.”²⁹ Wyplosz goes on to suggest that the process of assessing debt sustainability is “mission impossible,” a fool’s errand of sorts that produces more errors even as it seeks to become more attuned to the complexities of the task.

These more methodological problems have also led to failures of performance and performativity. Most of the World Bank staff I spoke with agreed that the social risk framework was a powerful set of ideas – but that ultimately it had not been effectively put into practice.³⁰ This is also the conclusion reached in the 2011 concept note prepared as part of the development of a new social protection strategy (SPS), which suggests that while the social risk management framework “has provided intellectual coherence” it has nonetheless “not always provided sufficient guidance to operations.”³¹ Even in cases where policies have been adapted to take more account of social risk, the evidence of success is mixed. The conditional cash transfers (CCTs), which are generally viewed as one of the biggest success stories of development policy innovation over the past decade, have had significant success in changing medium-term behaviour – e.g. increasing the number of days children stay in school. Yet there is little evidence of real improvement in “final” outcomes – such as educational attainment or better wages.³² It is not clear that efforts to “make up” new kinds of risk-managing and long-term thinking individuals – the ultimate goal of these performative policies – have yet succeeded.

Measuring results

Although the results agenda has come a long way over these past two decades, the shift towards results-driven policy has not been smooth, and the strategy remains far from black-boxed. There is no question that there is significant momentum towards results measurement among donor agencies and the IFIs; in the words of one senior IMF staff member, there has been “huge pressure” on both the Fund and the World Bank (particularly from the UK and the US) to adopt more results-oriented policies.³³ Yet there has also been some push-back from the IFIs – most notably by the IMF, but also, to a lesser extent, by World Bank staff.

One place where this push-back is particularly evident is at the IMF. Fund staff have actively resisted moving in the direction of measuring results because they see their policy practices as defying straightforward causal analysis of the kind required for results management. The Fund staff members I spoke with argued that the desired results of Fund programs – such as lowering inflation or increasing economic growth in a given country – are so broad and affected by so many factors that it is impossible to pinpoint the effects of a given Fund program.³⁴ As one senior staff member put it, the causes of something like economic growth are “somewhat mysterious,” making it unlikely that the Fund would ever embrace the results agenda.³⁵

This is not the first time that the IMF has considered and ultimately stepped back from placing results evaluation at the heart of their policies. The IMF Executive Board discussed the possibility of integrating results more directly into their lending programs in the context of their 2002 review of conditionality. At that time, they discussed the possibility of introducing outcomes-based conditionality, in which funds would be disbursed once a country met agreed-upon outcomes.³⁶ Yet the Board was divided on the issue and the staff ultimately recommended against adopting the measures – partly because they could not see how the Board could walk away from a program if it failed due to external factors.³⁷

It is not just the IMF, however, that has been resisting the results agenda. Andrew Natsios, who headed up the US Agency for International Development (USAID) from 2001 to 2005, has published a widely read critique of the perverse effects of results-based management and other examples of what he calls “obsessive measurement disorder.”³⁸ Some World Bank staff members have also been pushing back against the pressure to focus on results. Not surprisingly, the Bank staff members’ responses have been mixed: those working in the social and human development sectors have been most supportive of the move towards measuring results, while others have been more resistant. As one senior staff member put it, some sectors have a hard time agreeing on the results that they want to obtain.³⁹ Moreover, some issues, such as governance and institution-building, are notoriously difficult to quantify, making the results agenda a harder sell.⁴⁰

This resistance on the part of certain staff members to the results agenda is linked to their perception that the strategy has failed to perform as promised. At the heart of the results agenda is the belief that it is possible to accurately identify the outcomes of particular IFI and donor policies. Yet even as staff at the World Bank, the IMF, the Canadian development department (CIDA), the British Department for International Development (DFID) and USAID are exhorted to demonstrate concrete results, many of them are finding it difficult to do so. Although IMF staff justify their resistance to results by arguing that broad macroeconomic factors like inflation levels are impossible to relate back to individual policies, it is just as difficult to determine the impact of a specific World Bank program on infant mortality rates. Both results are over-determined by an array of complex factors. Staff at these institutions are not unaware of these methodological difficulties and their practical implications. A number of Organisation of Economic Co-operation and Development (OECD) staff members I spoke with described this as the crucial problem of attribution: the virtual impossibility of knowing with any certainty to what degree one assistance program among many (in addition to numerous other variables) may have achieved a given outcome.⁴¹

Another major performative objective of the strategy of results measurement is to foster a results culture among IFI, donor- and borrowing-country bureaucracies. Yet here again, the results (so to speak) have been far from consistent. Although aid agencies and IFIs are supposed to publish their results and learn from their mistakes, a number of donor and IFI staff suggested to me that there is considerable institutional pressure to show successes rather than to admit failures. Others suggested that, at the World Bank at least, the dominant culture is still one of moving funds out of the door, with results assessment being largely treated as a hoop to jump through rather than a meaningful criterion.⁴² There is just too much temptation to massage results matrixes to show that things are going well. As Isaline Bergamaschi shows in the Malian case, developing country governments have also resisted or “appropriated” the results-based agenda in various ways, by, for example, setting indicators to relatively easily achieved levels.⁴³ Even where Malian officials have had to cede ground to lenders’ concerns, they have seen the results strategy as an instrument of control, undermining the kind of entrepreneurial bureaucratic culture that IFIs and donors had hoped to create.⁴⁴

Two possible directions

There are two directions that this more provisional style of governance can potentially take in its effects on political life: one is more experimental, democratic and open-ended, while the other is more cautious, instrumentalizing and even cynical. Moreover, although both remain open as possible paths, the particular form that policy failures have taken has tended to reinforce this second, darker form of provisional governance.

One possible definition for the term “provisional” focuses on its temporary and experimental qualities: in contrast with a more confident governance style, a provisional approach is less certain about the outcomes and therefore potentially more open to trying different things.⁵³ We can see this more experimental sensibility in the various strategies discussed in this book. In different ways and to different extents, these new strategies are attuned to the complexity of the task at hand, recognizing the political, social and cultural dimensions of economic development. This messier ontology of development finance can lead to a greater awareness, even humility, about the limits of IFIs’ expertise – as when the IMF admits that it cannot realistically predict the next crisis but must instead settle for identifying potential underlying vulnerabilities, or when both IFIs and donors recognize the difficulties of defining and operationalizing a concept as subjective as political ownership.

In spite of, or perhaps because of, this greater uncertainty, institutional actors and groups have sometimes taken greater risks by trying new, more experimental ways of achieving their objectives – echoing trends in other fields that have been identified by Charles Sabel, Jonathan Zeitlin and Paul Rabinow.⁵⁴ The World Bank’s demand for good governance (DFGG) initiative is an excellent example of this trial and error approach, as its advocates move into new terrains and try numerous options, not all of which will necessarily succeed.⁵⁵ On a less ambitious scale, the IMF’s Executive Board also developed several experimental case studies to test out its standards and codes initiative before formalizing the policy. This kind of openness is also linked to a willingness to learn from mistakes. For example, in its early days, results-based management (RBM) was all about developing a more responsive and reflexive organization that tried new things and learned from its mistakes. More participatory forms of measurement and evaluation in particular left themselves open to surprises by giving local communities a say in judging success and failure, and early debates about RBM emphasized the importance of maintaining this more experimental and open-ended approach.⁵⁶ In this same spirit, the current World Bank President, Jim Yong Kim, has embraced the practice of “Fail Faires” that encourage NGO and IFI staff to acknowledge, discuss and learn from recent failures.⁵⁷

We can also find this openness to local forms of knowledge and expertise in many of the other governance strategies: techniques like the PRSP and DFGG are designed to shift some of the authority for identifying development priorities to local actors. Insofar as it opens expertise up to a wider range of actors, this more experimental sensibility resonates with what Callon and others have described as a more “technical democracy,” in which the division between professionals and lay people is broken down and everyday experimentation supplements more traditional forms of knowledge creation.⁵⁸ There thus exists within this turn to a more provisional kind of governance the potential for a movement towards not just a more experimental kind of expertise, but also a more open-ended form of politics.

Although this more open-ended sensibility has played an important role in shaping the strategies discussed in this book, it exists alongside a narrower, risk-averse kind of governance. Paradoxically, the term “provisional” can suggest both an experimental approach and a far more cautious one. It is this second meaning of “provisional” that Niklas Luhmann draws on when he suggests that those using provisional forms of expertise make claims that can be reversed or revised without losing face.⁵⁹ As I have suggested in my examination of the four new governance strategies, each uses certain techniques to hedge against the possibility of failure – through results matrixes that can be fudged after the fact, risk assessments that refuse to make definitive predictions, and conceptions of ownership and global standards that rely more on symbolic appearance (for the sake of credibility) than reality. These techniques are not experimental or open-ended: their goal is to avoid risks by carefully controlling the form and flow of knowledge.

We can see this more cautious side of provisional governance in the persistent desire of IFIs and donor agencies to quantify everything – good governance, ownership, vulnerabilities and results. Numbers provide a sense of epistemological security to bureaucrats whose expertise has traditionally relied on quantitative assessments. Although this desire to quantify is often qualified by recognition of the limits to doing so, we have witnessed an increasing trend towards attempting to quantify the unquantifiable. The results agenda, which is the newest of the four strategies discussed here, epitomizes that tendency, particularly in recent years as donor agencies have been pushed to translate even the most complex of policies into quantitative results in order to have some “hard data” on policy successes.⁶⁰ The current Bank President Jim Yong Kim’s recent emphasis on the need to create a “science of delivery” underpinned by “a relentless drive for results” is just one more example of this belief that quantification is the best way of avoiding future failure.⁶¹

Political implications

This trend towards the more cautious form of provisional governance has some significant political consequences. Since not everything can be turned into numbers, something has to give. As one OECD staff member put it, “with an extreme focus on bean counting now, there might be a tendency to move away from the more difficult areas like institution building and governance.”⁶² Yet these institutions are still engaged in the messy world of political and social transformation through their emphasis on good governance, poverty reduction plans and social risk. How do they make these complex and often-politicized programs consistent with the twin desires to protect their expertise and minimize risk? They do it by simplifying and instrumentalizing politics: translating political ideas such as consent, participation, difference and accountability into concepts that are consistent with economics (or, more specifically, public choice theory).

In Discourses of Power, Barry Hindess draws on Foucault’s later work to provide an analysis of the logic of contemporary liberal governance. He suggests that juridical power in a liberal political system emphasizes the importance of legitimate consent. In contrast, governmental power – or governmentality – works through technical and often non-state mechanisms to achieve its ends. It is a kind of power that avoids the question of consent by depoliticizing its exercise as a technical matter.⁶³ Yet if government is a form of power that avoids the problem of consent, then why has obtaining popular support become one of the major objectives of many of these new governmental strategies? Perhaps these strategies point to a further evolution in the logic of liberal governance: a move to make consent itself a kind of technique.

This instrumental approach to consent is apparent in policies focused on fostering demand for good governance. The assumption here is that if you provide civil society with enough information about the government’s actions, they will demand the same kinds of governance reforms that the World Bank and donors (who are the supply side of this equation) would like to introduce. This is a conception of politics that assumes that there is little variation in the kinds of demands that will be made by an informed and organized public. All will want the same kind of liberal, market-friendly good governance as the IFIs. While policies such as streamlined conditionality and the PRSP seek to respond to the particularities of different national and local contexts, they thus rely on a shallow conception of the scope of that difference. If this is the case, then the scope of political debate can remain quite narrow, involving consultation, report cards and limited participation rather than genuine deliberation. Encouraging local participation need not complicate the imperative to pursue the “correct” development path.

It is not just consent and difference that often end up being instrumentalized in such policies: accountability, one of the watch-words of so many of these new strategies, is often reduced to something akin to a market transaction. When IFI and donor staff encourage governments to publish their compliance with standards and codes or prepare report cards on their development results, it is in order to improve borrowing governments’ accountability to market and civil society actors. Although this objective is admirable, too often the accountability relationship between government and citizen is seen through the lens of service provision and understood as equivalent to that between a seller and a consumer.⁶⁴ While there is certainly enormous room in many countries for better “consumer satisfaction” when it comes to government services, this is not the same thing as democratic accountability. Genuine political accountability requires a more robust and wide-ranging kind of dialogue between government representatives and citizens, in which more fundamental priorities are on the table, not just feedback on a narrow set of services or the limited social priorities up for debate in the PRSP consultations.

Instrumental conceptions of consent, difference, participation and accountability allow IFI and donor actors to engage with the complexities of politics while still translating them (more or less) into the language of economic expertise. This methodologically cautious approach to governance conceals the power dynamics and inequalities at work – the differential levels of access various groups have to participatory processes in the case of the PRSP consultation processes, the many issues that are not up for discussion, and the forms of exclusion that are enabled through the processes of sorting and ranking borrowing countries that inscriptions like the ROSCs and DSAs make possible. This more cautious approach to provisional governance thus rests on a kind of impoverished fantasy about the nature of political debate. Unfortunately, it is a fantasy that has some potentially serious consequences for those at the receiving end of development finance.

Where are we heading?

Both experimental and cautious forms of provisional governance remain present in current governance strategies, and there remain a good number of actors – certain groups of IFI and donor staff, NGOs and scholars – who continue to push to have these new governance strategies meet their democratic potential. Yet there are several reasons to be less than optimistic about the direction that this new style of provisional governance is currently taking.

If we look back at the various policy failures discussed above, the future of provisional governance becomes a little less bright. Many of these failures are related to those aspects of governance strategies that are most innovative and potentially experimental, such as the participatory processes that have turned out to be thin and instrumentalized. Moreover, staff often view the difficulties involved in translating these more participatory processes into operational policies as a sign of the limits of such experiments – hence the growing discomfort about long and unprioritized PRSPs. Other policies, like results-based measurement, that were once more experimental have become far more cautious, even cynical, as bureaucrats have found ways of fudging the results.

Even when these more cautious techniques have themselves failed, as in the many methodological problems encountered when institutional actors seek to translate complex issues into tidy numbers, the response by staff and management has rarely been to recognize the limits of quantification. Instead, the recent trend has been to insist even more vigorously on the importance of quantitative data, while also noting possible methodological problems as a hedge against failure. How failures matter, once again, depends on how they are interpreted. To date, IFI and donor actors have tended to interpret recent failures in a way that reinforces the move to more cautious forms of governance.

This tendency to adopt a more risk-averse style of governance is nowhere clearer than in the new push for “value for money” (VFM), an approach that has become a mantra in the context of the recent global financial crisis. Those seeking VFM demand that aid dollars be rigorously justified in terms of their “efficiency, economy and effectiveness.”⁶⁵ This ultimately means not only assessing the costs of a given program but also translating its effects into quantitative results. The VFM approach is particularly popular among the recently elected conservative governments in the UK, Canada and Sweden. Once bastions of more progressive approaches to development assistance, many of these countries have begun to place more emphasis on the political dimensions of development finance.⁶⁶ Although we might expect that this increasing politicization of aid would significantly erode the provisional approach to governance, these governments have embraced many of the strategies examined in this book – the pursuit of global standards, risk management and, above all, the results agenda. Their push for VFM has played a key role in moving the results agenda even further towards a fixation on quantitative and standardized measurements.⁶⁷

This obsession with measuring the immeasurable pushes the cautious tendencies of current development governance to their logical extreme, effectively transforming development results from symbolic representations into nearly fictitious inventions. While such inventions are of little use to local populations in low-income countries (LICs) who might be interested in knowing which policies have been working and which have not, they do nonetheless serve the purpose they are increasingly designed for: public relations. After all, the primary audience for many of these quantitative results matrixes and reports is not aid recipients at all, but the donors – or, more specifically, the voting public. This donor-driven approach to results is not particularly good for the quality of political dialogue either in low-income countries or in donor communities. Nor is it good for the lending agencies themselves.

Why would this shift to a more cautious kind of governance endanger the institutions that practice it? As I argued at the very beginning of this book, the authority of global governance institutions relies heavily on their claims to expertise. As this book has demonstrated, that expertise is contestable and malleable, taking different forms in different eras. Yet it is also always fragile, resting on a set of epistemological and ontological assumptions that do not always fit the messiness of the world that it seeks to understand. While the shift to provisional governance has given development institutions a more dynamic ontology and granted them a measure of additional epistemological flexibility, IFIs’ and donors’ increasing caution paradoxically threatens to undercut their resilience. When the deniability of expert claims is pushed towards parody, as staff are pressured to interpret even the most unpromising of results as successes, this kind of caution becomes self-defeating. It is all too easy to realize that the numbers are fudged, that the scores are arbitrary, and that there is no referent behind the signal. If that happens, institutions will once again lose the authority that they sought so carefully to rebuild, and we will find ourselves back where we started: decrying the failure of development finance and trying to figure out where to go next.