March 20, 2023

The World Bank’s evolution is a large part of the international response to global challenges like climate change and pandemic risks, with significant attention on amounts and sources of money. But less attention is paid to an inconvenient truth: few policymakers and experts know what works to make measurable progress against global challenges. On climate and development, for example, knowledge is sparse; demonstrating that a project or policy’s climate impact is “real, measurable and additional” remains a work in progress.

Generating and using evidence is imperative to enhance both the development and the global impact of public and aid spending. There has been significant improvement on the development side, including new data and methods and a growing global community of evaluators. But still, only a small share of development programs is rigorously evaluated.

With global challenges high on the international agenda and financing set to increase in the coming years, a big push on evidence, backed by dedicated resources, is essential. The potential rate of return is immense: better data on results could save hundreds of millions in mistargeted or ineffective spending—and importantly, could reduce potential trade-offs with funding for poverty reduction and development more generally.

This note makes the case for a reinvigorated evidence agenda to boost the impact of financing for development and global public goods, recognizing synergies and trade-offs between these dual goals.

Global action, local impact: Evidence for development

Generating and using evidence to improve programs and public policies is good value for money. Several examples demonstrate the impact of evidence on lives improved and money saved across sectors and regions.

Our recent CGD Working Group on New Evidence Tools for Policy Impact spotlighted areas of significant progress in the evaluation and evidence ecosystem to better harness the value of evidence, including growing interest from policymakers in using evaluation evidence to inform programs and policies; a growing global community of researchers, organizations, and partnerships conducting evaluations; and advances in data and methodologies that enable faster, lower-cost studies.

Still, major challenges persist. Across government, aid agencies, and other international organizations, funding, capacity, and institutional incentives for evidence generation and use remain limited

As a result, we know surprisingly little about the results of spending on development. Only 10 percent of evaluations conducted or commissioned by bilateral agencies like USAID and the German Institute for Development Evaluation are impact evaluations. And less than 5 percent of World Bank projects since 2010 have been subject to impact evaluation. While not every policy or project requires or would benefit from a formal impact evaluation, more representative evidence on the effectiveness of operations is needed given the huge opportunity costs associated with less cost-effective uses of scarce development, global health, and climate spending

For instance, the best social interventions are 10 to 100 times more cost-effective than the average interventions; these differences generally hold up across global health, education, climate change, and other social interventions and apply in rich and poor countries alike. For climate mitigation specifically, the (limited) data we have so far suggests the most cost-effective interventions could be 260 times as effective as the least effective ones.  

CGD’s Working Group on New Evidence Tools for Policy Impact offers specific recommendations to enhance the policy value and use of this type of rigorous evidence for global development, including designing evaluations that start from the policy question and decision space available, advancing locally grounded evidence-to-policy partnerships to develop and shape these studies, and enhancing incentives to strengthen use of such evidence. 

Local action, global impact: Evidence for global public good

The recent spotlight on global challenges comes with a renewed imperative to harness results and evidence for greater impact. While there are conceptual and measurement issues that will need to be tackled (more below), we still know far too little about how effective spending on global challenges is. 

“Global public goods” (GPG) refer to institutions, mechanisms, and outcomes that benefit people across country income groups and extend to both current and future generations. In economic terms, they are considered non-rival and non-excludable, meaning one country’s benefit does not affect or exclude benefit by others. CGD and others have long discussed the role of multilateral institutions like the World Bank in taking a GPG approach to tackling global challenges like climate change and pandemic preparedness, which defy borders and disproportionately affect the poorest and most vulnerable

Not all GPG efforts can—or should—be rigorously evaluated, and the social benefits of additional information on some specific areas and programs are likely to be much greater than others. Both policymakers and funders should make strategic and intentional decisions on when to invest in and collect more evidence, both for GPGs and development programs. Our working group report suggests using a “value of information” approach to proactively consider and prioritize evaluation investments with the greatest potential social returns, including in areas that receive substantial resources and have a sparse evidence base.

Areas like climate mitigation and pandemic preparedness represent large and growing areas of concessional finance. Filling substantial knowledge gaps in these areas would address high-value decision-making needs. But doing so will require additional dedicated resources for data and evidence-related activities, including both generating individual studies and using bodies of evidence that bring together different studies and types of data.

Climate mitigation

Mitigation has hugely consequential global benefits. One ton of greenhouse gas reduction or removal anywhere has impact everywhere, meaning that the local benefit of mitigation is a tiny fraction of the global shared benefit. Many middle-income countries have growing emissions, and while the specific outcome trade-offs of using public expenditure and aid on climate mitigation versus other areas are important to consider, they remain understudied.  

Climate finance provided and mobilized for low- and middle-income countries totaled over $83 billion in 2020, with over 80 percent in loans and grants from public budgets. Climate-relevant projects currently account for one-third of bilateral ODA, and recent assessments suggest total financing needs for climate in the trillions of dollars. And as mentioned, the World Bank continues to rethink its mandate and tools to effectively deploy more climate finance. According to a new CGD paper, climate mitigation finance from the Climate Investment Funds and Green Climate Fund has generally been channeled to the countries and sectors with the highest emissions. But resource allocation across the largest recipients is not fully aligned with the size of country emissions, nor does allocation seem to optimize for additionality and catalytic impact with these concessional resources. Bringing more and better data and evidence to bear on resource allocation decisions could yield substantial benefits.

Yet, as previous CGD research has shown, we know surprisingly little about the effectiveness of spending on climate. For instance, out of 10,000 impact evaluations in 3iE’s database, less than 120 were tagged as “climate adaptation,” “climate mitigation,” and “climate resilience” (see figure). Cost-effectiveness evidence is also scant. What little is known shows highly variable results and cost-effectiveness estimates. A recent audit found that the European Commission’s 15-year effort to help 80 countries address problems from climate change had no demonstrable impact on countries’ climate resilience. And using the Green Climate Fund’s ex-ante estimates of project cost-effectiveness, the most cost-effective programs prevent over 200 times more emissions than the least effective. Despite the huge implications of these differences, we have far too little evidence to draw from to assess the climate impact of development projects or the development impact of climate projects. 

Stepping back, we acknowledge that defining and measuring mitigation externalities and marginal benefits involves underlying complexities and is not yet well developed. But more should be done to strive for measurement rigor; empirically evaluating cost-effectiveness or cost-benefit should be a priority, not just focusing on semi-artificial or ex-ante results measurement. Given the common challenges and measures of success, evaluation efforts should also see substantial collaboration.

Number of impact evaluations recorded by key words

Source: Reproduced from Cichocka and Mitchell, 2022 “Climate Finance Effectiveness: Six Challenging Trends” CGD Policy Paper 281. 

Pandemic preparedness

The next pandemic is a matter of when, not if. But because the health and economic consequences are felt differentially across countries, the return on investment in pandemic preparedness varies significantly. Again, while the trade-offs related to different uses of public expenditure and/or aid are likely important, they are understudied. 

Development assistance for health in 2021 totaled $67 billion; “preparedness” historically has made up roughly 2-10 percent of development assistance for health, depending on definitions and how you slice and dice the data. With a new fund established at the World Bank and the annual funding gap for pandemic preparedness estimated at 10 billion, we should see an uptick in spending (beyond ODA) in the years to come. 

We have little evidence on the effectiveness of pandemic preparedness interventions and surveillance systems in different country contexts. What we have are lists of policies and actions (even indicators are not fully agreed upon). But what we need is clear evidence on a menu of “best buys” for preparedness, taking into account the notional capital and recurrent costs of various multi-component surveillance systems to assess how much health impact can be bought with a given system. The WHO’s recently launched Mosaic Respiratory Surveillance Framework, which can help national authorities identify priority surveillance objectives and the best approaches to meet and evaluate them, is a step in the right direction.

But given remaining gaps, resources are currently deployed in uncoordinated and fragmented ways. It is unclear what is working for what purpose to deliver better outcomes. Information on costs and benefits from different perspectives (national vs. regional vs. global; ex-ante and during) is not just “nice-to-have knowledge” but should inform real-world resource allocation, including at the newly created Pandemic Fund

Trade-offs and synergies

When it comes to generating evidence in these areas, there are important differences and trade-offs to consider. For instance, the ultimate outcomes we’re working towards through GPGs relate to avoiding worst case scenarios (i.e., an absence measured by proxies), which is fundamentally different from development outcomes. There are also unanswered questions around how to measure and account for the differences in local and global benefits of GPGs. Evaluations can also be used to help unpack distributional questions about to whom the benefits from these investments accrue

Across the board, more evidence can mean more impact. With the tools at our disposal, we should keep the focus on recognizing lessons from the past; building on progress; increasing funding; improving incentives; and recognizing the interconnectedness between development, climate mitigation, and preparedness goals. As more financing for global challenges is deployed in the coming years, data and evidence use is needed to shift resources towards the most effective approaches and deliver demonstrable results.

What next? A closer look at the World Bank

Policymakers and practitioners must act now to integrate evidence into their efforts to tackle global challenges—and evaluation and evidence funders and practitioners must also take action to integrate high-value GPG-related questions into their evidence agendas. 

For example, as the World Bank operationalizes its 2021 Strategic Framework for Knowledge and advances institution-wide reforms outlined in its evolution roadmap, all while preparing for a new president to take the helm, linking its agendas on global challenges and knowledge generation would help keep the focus on real-world impact. 

The Strategic Framework for Knowledge includes little mention of the evidence agenda for global challenges like pandemic preparedness and climate. But we are glad to see the evolution roadmap emphasize how the World Bank can “further strengthen its focus on outcomes by strengthening investment in data, impact evaluation and results architecture,” noting that impact evaluations are currently underutilized across the bank’s portfolio and that investment in data generation and capacity in developing countries is inadequate. The roadmap recognizes the need for increased investments in impact evaluation to guide domestic spending, inform operations supporting GPGs, and “enhance the quality of project design and implementation by informing evidence-based mid-course corrections and filling knowledge gaps that then benefit subsequent projects.

But translating these intentions into real progress is where the rubber meets the road. As part of CGD’s working group, we highlighted ways the World Bank can leverage knowledge generation for policy impact, including embedding impact evaluation and related evidence activities across its operational structure and developing sectoral, regional, and country learning agendas. And achieving meaningful progress will require additional resources be dedicated for evaluation and evidence activities, as noted in the roadmap.

By prioritizing even a small share of resources for evidence-related functions, the World Bank’s leadership and shareholders would assure that all resources are spent more effectively and efficiently. Specifically, the bank needs a range of evaluation structures to work with country partners to inform policy and lending operations to make timely adjustments and generate and sustain policymaker demand for evidence, while also protecting the integrity and independence of the research to remain high-quality and credible. In terms of resources, funding for impact evaluation currently depends on insufficient—and at times uncertain—external trust fund financing and fragmented operational interest. New resources are needed to undertake evaluations more strategically and systematically across the World Bank’s evolving portfolio.

In addition to these measures, the World Bank’s board and management could consider the following actions to finance evaluation and evidence production and use:

1. include attention to the dual agenda in data, research, and evaluation as part of organizational KPI revisions within the roadmap process;
2. set aside a defined allocation of new financing mobilized by the bank to address global challenges;
3. ensure all new GPG-focused projects have some evaluation resources attached, or ensure the top 10 percent (by value) of new GPG projects are evaluated; 
4. support trust funds to carry out longer-term research projects that focus on specific regions or global challenges and go beyond the timelines of specific operations; and 
5. embed knowledge management within the World Bank’s Governance Global Practice, which could then pursue joint ventures with other global practices as a way to contribute to strengthened in-country evidence functions and capacities, building on collaboration between DIME and the Global Governance Practice. 

Regardless of the financing mechanism, using evidence to inform investment—and disinvestment—decisions is our best shot at meeting the challenges of today and tomorrow. 

Learn more