Hostname: page-component-cd9895bd7-gxg78 Total loading time: 0 Render date: 2024-12-31T14:09:12.298Z Has data issue: false hasContentIssue false
  • English
  • Français

The challenge of monitoring and evaluation under the new aid modalities: experiences from Rwanda*

Published online by Cambridge University Press:  11 November 2008

Nathalie Holvoet  and
Heidy Rombouts
Nathalie Holvoet*
Affiliation:
Institute of Development Policy and Management, University of Antwerp, Lange St. Annastraat 7, 2000 Antwerpen, Belgium
Heidy Rombouts
Affiliation:
Belgian Technical Cooperation, Hoogstraat 147, 1000 Brussel, Belgium
*
Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Monitoring and evaluation (M&E) are sensitive issues in relations between donor agencies and recipient governments, especially in a time when the responsibility for implementing aid activities is shifting towards recipients. This paper deplores that, so far, donors and recipients have adopted an overly technocratic approach to M&E, largely disregarding broader institutional and systemic issues. Using case study material from Rwanda, we illustrate that assessments regarding the quality of a country's M&E efforts may differ sharply depending upon one's perspective. At the core of the matter is ‘the denial of politics’, one of the most serious flaws in the new aid paradigm promoted in the OECD's 2005 ‘Paris Declaration’. We argue that while a narrowly defined technocratic vision of M&E may seem ‘politically neutral’, in fact it may jeopardise M&E's functions of ‘accountability’ and ‘feedback’. This can eventually undermine the effective implementation of some of the key principles of the ‘new aid approach’.

Type
Research Article
Information
The Journal of Modern African Studies , Volume 46 , Issue 4 , December 2008 , pp. 577 - 602
Copyright
Copyright © 2008 Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

The authors thank the interviewees, anonymous referees and Robrecht Renard for useful comments and suggestions. The views expressed in this paper are those of the authors, and should not be attributed to either the Institute of Development Policy and Management, University of Antwerp, or the Belgian Technical Cooperation.

References

REFERENCES

Amis, P., Jackson, P. & Ahobamuteze, G.. 2005. ‘Baseline survey of donor alignment and harmonisation in Rwanda’, University of Birmingham, International Development Department.Google Scholar
Anon. 2005. ‘2004 Rwanda PRS Annual Progress Report: joint civil society comments on the zero Draft of 17th June 2005’, Kigali.Google Scholar
Armon, J. 2007. ‘Aid, politics and development: a donor perspective’, Development Policy Review 25, 5: 653–6.CrossRefGoogle Scholar
Bamberger, M. 1989. ‘The monitoring and evaluation of public sector programs in Asia: why are development programs monitored but not evaluated?’, Evaluation Review 13, 3: 223–42.CrossRefGoogle Scholar
Bamberger, M. 1991. ‘The politics of evaluation in development’, Evaluation and Program Planning 14: 325–39.CrossRefGoogle Scholar
Bamberger, M. 2000. ‘The evaluation of international development programs: a view from the front’, American Journal of Evaluation 21, 1: 95–102.CrossRefGoogle Scholar
Bedi, T., Coudouel, A., Cox, M., Goldstein, M. & Thornton, N.. 2006. Beyond the Numbers: understanding the institutions for monitoring poverty reduction strategies. Washington, DC: World Bank.Google Scholar
Booth, D. & Lucas, H.. 2002. ‘Good practice in the development of PRSP indicators and monitoring systems’, ODI working paper 172, London: ODI.Google Scholar
Booth, D. 2005. ‘Missing links in the politics of development: learning from the PRSP experiment’, ODI working paper 256, London: ODI.Google Scholar
Bratton, M. 1989. ‘The politics of government – NGO relations in Africa’, World Development 17: 569–87.CrossRefGoogle Scholar
Brown, G. & Stewart, F.. 2007. ‘The implications of horizontal inequalities for aid’, UNU-WIDER discussion paper 2007/51, Helsinki: UNU-WIDER.Google Scholar
Browne, S. 2007. ‘Aid to fragile states: do donors help or hinder?’, UNU-WIDER discussion paper 2007/01, Helsinki: UNU-WIDER.Google Scholar
Centre for Global Development (CGD), Evaluation Gap Working Group. 2006. When Will We Ever Learn? Improving lives through impact evaluation. Washington DC: CGD.Google Scholar
Chelimsky, E. 1987a. ‘What have we learned about the politics of program evaluation?’, Evaluation Practice 8, 1: 5–21.Google Scholar
Chelimsky, E. 1987b. ‘Linking program evaluation to user needs’, in Palumbo, D. ed. The Politics of Program Evaluation. London: Sage, 72–99.Google Scholar
Commission Parlementaire. 2004. Là où l'idéologie génocidaire se fait observer au Rwanda. Kigali: Commission Parlementaire.Google Scholar
Cordaid. 2005. Beyond Data: a panorama of CSO experiences with PRSP and HIPC monitoring. Den Haag: Cordaid.Google Scholar
Craig, D. & Porter, D.. 2003. ‘Poverty reduction strategy papers: a new convergence’, World Development 31, 1: 53–69.CrossRefGoogle Scholar
Development Gateway Foundation Inc. 2008. <www.aidharmonisation.org>>Google Scholar
Development Partners Coordination Group (DPCG). 2007. Rwanda Development Partners Website. <http://www.devpartners.gov.rw>>Google Scholar
Driscoll, R. & Evans, A.. 2005. ‘Second generation poverty reduction strategies: new opportunities and emerging issues’, Development Policy Review 23, 1: 5–25.CrossRefGoogle Scholar
Elkins, C. 2006. ‘Monitoring and evaluation (M&E) for development in peace-precarious situations’, paper prepared for the 47th Annual ISA Convention San Diego, CA.Google Scholar
Evans, A., Piron, L., Curran, Z. & Driscoll, R.. 2006. Independent Evaluation of Rwanda's Poverty Reduction Strategy 2002–2005 (PRSP1). London/Brighton: ODI/IDS.Google Scholar
Gordillo, G. & Andersson, K.. 2004. ‘From policy lessons to policy actions: motivation to take evaluation seriously’, Public Administration and Development 24, 4: 305–20.CrossRefGoogle Scholar
GTZ/BMZ (Deutsche Geschellschaft für Technische Zusammenarbeit/German Federal Ministry for Economic Cooperation and Development). 2004. National Monitoring of Strategies for Sustainable Poverty Reduction/PRSPs, vol. i. Main Report. Eshborn: GTZ.Google Scholar
Hayman, R. 2007. ‘Are the MDGs enough? Donor perspectives and recipient visions of education and poverty reduction in Rwanda’, International Journal of Educational Development 27: 371–82.CrossRefGoogle Scholar
Holvoet, N. & Renard, R.. 2007. ‘Monitoring and evaluation under the PRSP: solid rock or quicksand?’, Evaluation and Program Planning 30, 1: 66–81.CrossRefGoogle ScholarPubMed
IMF & World Bank. 2005. 2005 Review of the Poverty Reduction Strategy Approach: balancing accountabilities and scaling up results. Washington DC: IMF & World Bank.Google Scholar
Jordaan, E. 2006. ‘Inadequately self-critical: Rwanda's self-assessment for the African Peer Review Mechanism’, African Affairs 105, 420: 333–51.CrossRefGoogle Scholar
Kaufmann, D., Kraay, A. & Mastruzzi, M.. 2007. Governance Matters vi: governance indictors for 1996–2006. Washington, DC: The World Bank. <http://info.worldbank.org/governance/wgi2007>Google Scholar
Kanyarukiga, S., Van Der Meer, E., Paalman, M., Poate, D. & Schrader, T.. 2006. ‘Evaluation of DfID Country Programmes: Country Study Rwanda 2000–2005’, DfID Evaluation report EV660. London: DfID.Google Scholar
Killick, T. 2004. ‘Politics, evidence and the new aid agenda’, Development Policy Review 22, 1: 5–29.CrossRefGoogle Scholar
Kusek, J. & Rist, R.. 2004. Ten Steps to Results-based Monitoring and Evaluation System. Washington, DC: World Bank.CrossRefGoogle Scholar
‘Loi N°20/2000 du 26/07/ 2000 relative aux associations sans but lucrative’, Journal Officiel N°7 du 01/04/2001, 39–47.Google Scholar
Lucas, H., Evans, D. & Pasteur, K.. 2004. ‘Research on the current state of PRS monitoring systems’, IDS discussion paper 382, Brighton: IDS.Google Scholar
McAdam, D. 1999. Political Process and the Development of Black Insurgency 1930–1970, 2nd edn.Chicago, IL: University of Chicago Press.CrossRefGoogle Scholar
Marriage, Z. 2006. ‘Defining morality: DFID and the Great Lakes’, Third World Quarterly 27, 3: 477–90.CrossRefGoogle Scholar
Meyer, D. & Staggenborg, S.. 1996. ‘Movements, counter-movements and structure of political opportunity’, American Journal of Sociology 101: 1628–60.CrossRefGoogle Scholar
Ministry of Education, Government of Rwanda (MINEDUC). 2006. Revue Conjointe du Secteur de l'Éducation (JRES). Kigali: MINEDUC.Google Scholar
Ministry of Finance & Economic Planning, Government of Rwanda (MINECOFIN). 2005. Cabinet Paper on the National Poverty Reduction Strategy Monitoring and Evaluation Framework. Kigali: MINECOFIN.Google Scholar
MINECOFIN. 2006. Aid Effectiveness Report Rwanda: annual report. Kigali: MINECOFIN.Google Scholar
MINECOFIN, National Poverty Reduction Programme, UNR Butare and Ossrea Rwanda Chapter. 2002. Poverty Reduction Policies Relevance Test, vols i–v. Kigali: MINECOFIN, UNR Butare and Ossrea Rwanda Chapter.Google Scholar
Mutebi, F. G., Stone, S. & Thin, N.. 2001. Institutionalising the PRSP Approach in Rwanda (Chapter 8 of the PRSP Institutionalisation Study). Oxford: Oxford Policy Management.Google Scholar
Nairobi M&E Network. 2002. ‘The African Evaluation Guidelines, 2002: a checklist to assist in planning evaluations, negotiating contracts, reviewing progress and ensuring adequate completion of an evaluation’, Evaluation and Program Planning 25, 4: 481–92.CrossRefGoogle Scholar
National Institute of Statistics Rwanda (NISR). 2006. Preliminary Poverty Update Report: integrated living conditions survey 2005/06 (first draft for consultation). Kigali: NISR.Google Scholar
OECD/DAC. 2002. Key Terms in Evaluation and Results Based Management. Paris: OECD/DAC.Google Scholar
OECD/DAC. 2005. Paris Declaration on Aid Effectiveness. Paris: OECD/DAC.Google Scholar
OECD/DAC. 2007a. 2006 Survey on Monitoring the Paris Declaration: overview of the results. Paris: OECD/DAC. <http://www.oecd.org/dac/effectiveness/monitoring>>Google Scholar
OECD/DAC. 2007b. 2006 Survey on Monitoring the Paris Declaration: Rwanda chapter. Paris: OECD/DAC.Google Scholar
OECD/DAC. 2008. 2008 Survey on Monitoring the Paris Declaration. Paris: OECD/DAC. <http://www.oecd.org/dac/hlfsurvey>>Google Scholar
OSSREA Rwanda Chapter. 2006. Rwanda Citizens Report and Community Score Cards. Kigali: OSSREA Rwanda Chapter.Google Scholar
Palumbo, D. ed. 1987. The Politics of Program Evaluation. London: Sage.Google Scholar
Palumbo, D. 1987. ‘Politics and Evaluation’, in Palumbo, , The Politics of Program Evaluation, 12–46.Google Scholar
Patton, M. Q. 1987. ‘Evaluation's political inherency: practical implications for design and use’, in Palumbo, , The Politics of Program Evaluation, 100–45.Google Scholar
Picciotto, R. 2003. ‘International trends and development evaluation: the need for ideas’, American Journal of Evaluation 24, 2: 227–34.CrossRefGoogle Scholar
Pottier, J. 2006. ‘Land reform for peace? Rwanda's 2005 land law in context’, Journal of Agrarian Change 6, 4: 509–37.CrossRefGoogle Scholar
Prennushi, G., Rubio G., G. & Subbarao, K.. 2001. ‘Chapter 3: Monitoring and Evaluation+Annexes’, in World Bank, PRSP Sourcebook. Washington DC: World Bank, 107–30; 433–62.Google Scholar
Pritchett, L. 2002. ‘It pays to be ignorant: a simple political economy of rigorous program evaluation’, Policy Reform 5, 4: 251–69.CrossRefGoogle Scholar
Purcell, R., Dom, C. & Ahobamuteze, G.. 2006. Joint Evaluation of General Budget Support 1994–2004. Rwanda Country Report. Birmingham: International Development Department (IDD) and Associates.Google Scholar
Ravallion, M. 2006. ‘Looking beyond averages in the trade and poverty debate’, World Development 34, 8: 1374–92.CrossRefGoogle Scholar
Ravallion, M. 2007. ‘Inequality is bad for the poor’, in Jenkins, S. & Micklewright, J. eds. Inequality and Poverty Re-Examined. OxfordUniversity Press.Google Scholar
Renard, R. & Molenaers, N.. 2003. ‘Civil society participation in Rwanda's poverty reduction strategy’, IDPM discussion paper 2003–5. Antwerp: IOB.Google Scholar
Reyntjens, F. 2004. ‘Rwanda, ten years on: from genocide to dictatorship’, African Affairs 103: 177–210.CrossRefGoogle Scholar
Rombouts, H. 2004. Victim Organisations and the Politics of Reparation: a case study on Rwanda. Antwerp: Intersentia.Google Scholar
Strategic Partnership with Africa (SPA) – Budget Support Working Group (BSWG). 2005. ‘2004 Survey of the Alignment of Budget Support and Balance of Payments Support with National PRS Processes’, report by the BSWG co-chairs (final draft 7 February 2005).Google Scholar
Stewart, F. 2006. ‘Policies towards horizontal inequalities in post-conflict reconstruction’, UNU-Wider research paper 2006/149, Helsinki: UNU-Wider.Google Scholar
Tilly, C. 1978. From Mobilization to Revolution. Reading, MA: Addison-Wesley.Google Scholar
Uvin, P. 2001. ‘Difficult choices in the new post-conflict agenda: the international community in Rwanda after the genocide’, Third World Quarterly 22, 2: 177–89.CrossRefGoogle Scholar
Weiss, C. 1970. ‘The politicization of evaluation research’, Journal of Social Issues 26, 4: 57–68.CrossRefGoogle Scholar
Weiss, C. 1987. ‘Where politics and evaluation research meet’, in Palumbo, , The Politics of Program Evaluation, 47–69.Google Scholar
Weiss, C. 1999. ‘The interface between evaluation and public policy’, Evaluation 5, 4: 468–86.CrossRefGoogle Scholar
World Bank. 2000. Monitoring and Evaluation Capacities in Ghana: a diagnosis and proposed action plan. Washington, DC: World Bank.Google Scholar
World Bank. 2002. 2002 Annual Report on Evaluation Capacity Development. Washington, DC: World Bank, Operations Evaluation Department.Google Scholar
World Bank. 2003. Toward Country-Led Development: a multi-partner evaluation of the comprehensive development framework. Synthesis Report. Washington, DC: World Bank.Google Scholar
World Bank. 2005. Enabling Capacity to Achieve Results: 2005 comprehensive development framework progress report. Washington, DC: World Bank, Operations Evaluation Department.Google Scholar
World Bank. 2006. World Development Indicators Database 2006. Washington, DC: World Bank.Google Scholar
World Bank. 2008. Country Statistical Information Database. <http://go.worldbank.org/MUOSWQBHD0>>Google Scholar

Interviews

Staff members of national and international NGOs, Kigali, 11.5.2006, 15.5.2006, 17.5.2006, 18.5.2006, 19.5.2006.

Staff from various non-budgetary donors, Kigali, 12.5.2006, 15.5.2006, 17.5.2006.

Staff members of ministries, Kigali, 12.5.2006, 15.5.2006, 18.5.2006, 19.5.2006.