Author: Valencia M

The World Bank’s evolution is a large part of the international response to global challenges like climate change and pandemic risks, with significant attention on amounts and sources of money. But less attention is paid to an inconvenient truth: few policymakers and experts know what works to make measurable progress against global challenges. On climate and development, for example, knowledge is sparse; demonstrating that a project or policy’s climate impact is “real, measurable and additional” remains a work in progress.

Generating and using evidence is imperative to enhance both the development and the global impact of public and aid spending. There has been significant improvement on the development side, including new data and methods and a growing global community of evaluators. But still, only a small share of development programs is rigorously evaluated.

With global challenges high on the international agenda and financing set to increase in the coming years, a big push on evidence, backed by dedicated resources, is essential. The potential rate of return is immense: better data on results could save hundreds of millions in mistargeted or ineffective spending—and importantly, could reduce potential trade-offs with funding for poverty reduction and development more generally.

This note makes the case for a reinvigorated evidence agenda to boost the impact of financing for development and global public goods, recognizing synergies and trade-offs between these dual goals.

Global action, local impact: Evidence for development

Generating and using evidence to improve programs and public policies is good value for money. Several examples demonstrate the impact of evidence on lives improved and money saved across sectors and regions.

Our recent CGD Working Group on New Evidence Tools for Policy Impact spotlighted areas of significant progress in the evaluation and evidence ecosystem to better harness the value of evidence, including growing interest from policymakers in using evaluation evidence to inform programs and policies; a growing global community of researchers, organizations, and partnerships conducting evaluations; and advances in data and methodologies that enable faster, lower-cost studies.

Still, major challenges persist. Across government, aid agencies, and other international organizations, funding, capacity, and institutional incentives for evidence generation and use remain limited

As a result, we know surprisingly little about the results of spending on development. Only 10 percent of evaluations conducted or commissioned by bilateral agencies like USAID and the German Institute for Development Evaluation are impact evaluations. And less than 5 percent of World Bank projects since 2010 have been subject to impact evaluation. While not every policy or project requires or would benefit from a formal impact evaluation, more representative evidence on the effectiveness of operations is needed given the huge opportunity costs associated with less cost-effective uses of scarce development, global health, and climate spending

For instance, the best social interventions are 10 to 100 times more cost-effective than the average interventions; these differences generally hold up across global health, education, climate change, and other social interventions and apply in rich and poor countries alike. For climate mitigation specifically, the (limited) data we have so far suggests the most cost-effective interventions could be 260 times as effective as the least effective ones.  

CGD’s Working Group on New Evidence Tools for Policy Impact offers specific recommendations to enhance the policy value and use of this type of rigorous evidence for global development, including designing evaluations that start from the policy question and decision space available, advancing locally grounded evidence-to-policy partnerships to develop and shape these studies, and enhancing incentives to strengthen use of such evidence. 

Local action, global impact: Evidence for global public good

The recent spotlight on global challenges comes with a renewed imperative to harness results and evidence for greater impact. While there are conceptual and measurement issues that will need to be tackled (more below), we still know far too little about how effective spending on global challenges is. 

“Global public goods” (GPG) refer to institutions, mechanisms, and outcomes that benefit people across country income groups and extend to both current and future generations. In economic terms, they are considered non-rival and non-excludable, meaning one country’s benefit does not affect or exclude benefit by others. CGD and others have long discussed the role of multilateral institutions like the World Bank in taking a GPG approach to tackling global challenges like climate change and pandemic preparedness, which defy borders and disproportionately affect the poorest and most vulnerable

Not all GPG efforts can—or should—be rigorously evaluated, and the social benefits of additional information on some specific areas and programs are likely to be much greater than others. Both policymakers and funders should make strategic and intentional decisions on when to invest in and collect more evidence, both for GPGs and development programs. Our working group report suggests using a “value of information” approach to proactively consider and prioritize evaluation investments with the greatest potential social returns, including in areas that receive substantial resources and have a sparse evidence base.

Areas like climate mitigation and pandemic preparedness represent large and growing areas of concessional finance. Filling substantial knowledge gaps in these areas would address high-value decision-making needs. But doing so will require additional dedicated resources for data and evidence-related activities, including both generating individual studies and using bodies of evidence that bring together different studies and types of data.

Climate mitigation

Mitigation has hugely consequential global benefits. One ton of greenhouse gas reduction or removal anywhere has impact everywhere, meaning that the local benefit of mitigation is a tiny fraction of the global shared benefit. Many middle-income countries have growing emissions, and while the specific outcome trade-offs of using public expenditure and aid on climate mitigation versus other areas are important to consider, they remain understudied.  

Climate finance provided and mobilized for low- and middle-income countries totaled over $83 billion in 2020, with over 80 percent in loans and grants from public budgets. Climate-relevant projects currently account for one-third of bilateral ODA, and recent assessments suggest total financing needs for climate in the trillions of dollars. And as mentioned, the World Bank continues to rethink its mandate and tools to effectively deploy more climate finance. According to a new CGD paper, climate mitigation finance from the Climate Investment Funds and Green Climate Fund has generally been channeled to the countries and sectors with the highest emissions. But resource allocation across the largest recipients is not fully aligned with the size of country emissions, nor does allocation seem to optimize for additionality and catalytic impact with these concessional resources. Bringing more and better data and evidence to bear on resource allocation decisions could yield substantial benefits.

Yet, as previous CGD research has shown, we know surprisingly little about the effectiveness of spending on climate. For instance, out of 10,000 impact evaluations in 3iE’s database, less than 120 were tagged as “climate adaptation,” “climate mitigation,” and “climate resilience” (see figure). Cost-effectiveness evidence is also scant. What little is known shows highly variable results and cost-effectiveness estimates. A recent audit found that the European Commission’s 15-year effort to help 80 countries address problems from climate change had no demonstrable impact on countries’ climate resilience. And using the Green Climate Fund’s ex-ante estimates of project cost-effectiveness, the most cost-effective programs prevent over 200 times more emissions than the least effective. Despite the huge implications of these differences, we have far too little evidence to draw from to assess the climate impact of development projects or the development impact of climate projects. 

Stepping back, we acknowledge that defining and measuring mitigation externalities and marginal benefits involves underlying complexities and is not yet well developed. But more should be done to strive for measurement rigor; empirically evaluating cost-effectiveness or cost-benefit should be a priority, not just focusing on semi-artificial or ex-ante results measurement. Given the common challenges and measures of success, evaluation efforts should also see substantial collaboration.

Number of impact evaluations recorded by key words

Source: Reproduced from Cichocka and Mitchell, 2022 “Climate Finance Effectiveness: Six Challenging Trends” CGD Policy Paper 281. 

Pandemic preparedness

The next pandemic is a matter of when, not if. But because the health and economic consequences are felt differentially across countries, the return on investment in pandemic preparedness varies significantly. Again, while the trade-offs related to different uses of public expenditure and/or aid are likely important, they are understudied. 

Development assistance for health in 2021 totaled $67 billion; “preparedness” historically has made up roughly 2-10 percent of development assistance for health, depending on definitions and how you slice and dice the data. With a new fund established at the World Bank and the annual funding gap for pandemic preparedness estimated at 10 billion, we should see an uptick in spending (beyond ODA) in the years to come. 

We have little evidence on the effectiveness of pandemic preparedness interventions and surveillance systems in different country contexts. What we have are lists of policies and actions (even indicators are not fully agreed upon). But what we need is clear evidence on a menu of “best buys” for preparedness, taking into account the notional capital and recurrent costs of various multi-component surveillance systems to assess how much health impact can be bought with a given system. The WHO’s recently launched Mosaic Respiratory Surveillance Framework, which can help national authorities identify priority surveillance objectives and the best approaches to meet and evaluate them, is a step in the right direction.

But given remaining gaps, resources are currently deployed in uncoordinated and fragmented ways. It is unclear what is working for what purpose to deliver better outcomes. Information on costs and benefits from different perspectives (national vs. regional vs. global; ex-ante and during) is not just “nice-to-have knowledge” but should inform real-world resource allocation, including at the newly created Pandemic Fund

Trade-offs and synergies

When it comes to generating evidence in these areas, there are important differences and trade-offs to consider. For instance, the ultimate outcomes we’re working towards through GPGs relate to avoiding worst case scenarios (i.e., an absence measured by proxies), which is fundamentally different from development outcomes. There are also unanswered questions around how to measure and account for the differences in local and global benefits of GPGs. Evaluations can also be used to help unpack distributional questions about to whom the benefits from these investments accrue

Across the board, more evidence can mean more impact. With the tools at our disposal, we should keep the focus on recognizing lessons from the past; building on progress; increasing funding; improving incentives; and recognizing the interconnectedness between development, climate mitigation, and preparedness goals. As more financing for global challenges is deployed in the coming years, data and evidence use is needed to shift resources towards the most effective approaches and deliver demonstrable results.

What next? A closer look at the World Bank

Policymakers and practitioners must act now to integrate evidence into their efforts to tackle global challenges—and evaluation and evidence funders and practitioners must also take action to integrate high-value GPG-related questions into their evidence agendas. 

For example, as the World Bank operationalizes its 2021 Strategic Framework for Knowledge and advances institution-wide reforms outlined in its evolution roadmap, all while preparing for a new president to take the helm, linking its agendas on global challenges and knowledge generation would help keep the focus on real-world impact. 

The Strategic Framework for Knowledge includes little mention of the evidence agenda for global challenges like pandemic preparedness and climate. But we are glad to see the evolution roadmap emphasize how the World Bank can “further strengthen its focus on outcomes by strengthening investment in data, impact evaluation and results architecture,” noting that impact evaluations are currently underutilized across the bank’s portfolio and that investment in data generation and capacity in developing countries is inadequate. The roadmap recognizes the need for increased investments in impact evaluation to guide domestic spending, inform operations supporting GPGs, and “enhance the quality of project design and implementation by informing evidence-based mid-course corrections and filling knowledge gaps that then benefit subsequent projects.

But translating these intentions into real progress is where the rubber meets the road. As part of CGD’s working group, we highlighted ways the World Bank can leverage knowledge generation for policy impact, including embedding impact evaluation and related evidence activities across its operational structure and developing sectoral, regional, and country learning agendas. And achieving meaningful progress will require additional resources be dedicated for evaluation and evidence activities, as noted in the roadmap.

By prioritizing even a small share of resources for evidence-related functions, the World Bank’s leadership and shareholders would assure that all resources are spent more effectively and efficiently. Specifically, the bank needs a range of evaluation structures to work with country partners to inform policy and lending operations to make timely adjustments and generate and sustain policymaker demand for evidence, while also protecting the integrity and independence of the research to remain high-quality and credible. In terms of resources, funding for impact evaluation currently depends on insufficient—and at times uncertain—external trust fund financing and fragmented operational interest. New resources are needed to undertake evaluations more strategically and systematically across the World Bank’s evolving portfolio.

In addition to these measures, the World Bank’s board and management could consider the following actions to finance evaluation and evidence production and use:

1. include attention to the dual agenda in data, research, and evaluation as part of organizational KPI revisions within the roadmap process;
2. set aside a defined allocation of new financing mobilized by the bank to address global challenges;
3. ensure all new GPG-focused projects have some evaluation resources attached, or ensure the top 10 percent (by value) of new GPG projects are evaluated; 
4. support trust funds to carry out longer-term research projects that focus on specific regions or global challenges and go beyond the timelines of specific operations; and 
5. embed knowledge management within the World Bank’s Governance Global Practice, which could then pursue joint ventures with other global practices as a way to contribute to strengthened in-country evidence functions and capacities, building on collaboration between DIME and the Global Governance Practice. 

Regardless of the financing mechanism, using evidence to inform investment—and disinvestment—decisions is our best shot at meeting the challenges of today and tomorrow. 

The World Bank’s Evolution Roadmap suggests the institution could provide more grants and subsidies to activities that support the provision of global public goods (GPGs), particularly in richer developing countries. Provision of those public goods is surely a priority: avoiding future pandemics and limiting climate change benefits everyone. At the same time, and especially in an environment where finance for international subsidies and grants is extremely limited, it is important that the mechanisms are efficient. That isn’t at all true of existing funding, and there is no agreed pathway to fix that problem. Donors have already spent $48 billion on climate and environment funds housed at the World Bank alone.  Before ramping up that spending, it is a financial and moral imperative to find an investment model that is fit for purpose.

The World Bank Group’s largest existing grant and subsidy mechanism has a simple but highly effective strategy for maximizing the potential impact of cheap or free money: deliver it to countries that don’t have very much money of their own. The World Bank’s IDA arm provides financing almost exclusively to countries with a GNI per capita of below $1,255. Giving aid to poorer countries maximizes the impact of each dollar spent in terms of impact on potential economic growth, lifting people out of poverty or improving health outcomes. World Bank oversight helps reassure donors that the resources are used relatively effectively towards those ends.

We lack a similar simple but effective filtering mechanism to ensure GPG financing is spent well.  In theory, there is a straightforward approach with climate mitigation, at least: use a carbon market. Indeed, if there was a global carbon trading scheme, we wouldn’t need the World Bank to be involved in climate mitigation financing at all. Sadly, however, we’re some way from trading carbon credits like we trade copper or iPhones.

In the absence of a global carbon market, the World Bank has supported ersatz equivalents.  The results have not been reassuring.  The (incomplete) failure of the Clean Development Mechanism as a carbon credit trading system came alongside considerable doubts about its certification mechanism when it comes to demonstrating additionality (that carbon credits were really associated with reduced greenhouse gas output). Perhaps the World Bank could agree to purchase carbon credits from one of the increasing number of national and regional carbon credit markets, if they passed certain standards. But, as with global carbon markets, it is questionable why donors would need the World Bank at all to support such purchases.

Meanwhile, the Green Climate Fund and Clean Technology Fund both apparently back projects that (potentially) reduce greenhouse gas (GHG) emissions at a cost per tonne that varies over two orders of magnitude, and that leaves aside many additionality concerns. Bespoke project-level mechanisms that try to provide the minimum subsidy to generate maximum GHG reduction are not only bureaucratically immensely complex, it isn’t clear they work (something we’ve seen with efforts to subsidize private sector projects in low income countries, as well).  The proliferation of climate funds is itself a further measure of the lack of consensus over how to efficiently allocate grants to reduce emissions. Before expanding such funds, the Bank should at least try to develop the knowledge base and methods that ensure grantmaking is cost effective (low-dollar investment cost per ton of greenhouse gasses averted, which is comparatively straightforward) and additional (evidence that the investment would not have taken place without the financing, which is considerably less straightforward).

Perhaps there is a role for the World Bank to support the interventions identified through its Country Climate and Development Reports, providing more generous terms as part of policy-based lending in support of the priority actions identified in those studies. While that wouldn’t maximize global GHG reductions per dollar spent, at least it would provide some level of reassurance that projects would be beneficial for both climate and development.  But simply providing a lower rate of interest to any World Bank loan that looks to be in support of a lower carbon development path raises considerable issues: available subsidy finance will be a drop in the bucket compared to the total investments required in client countries, and this approach will be poorly targeted to support the investments that actually need a subsidy to go ahead.  It will provide cheap money to some emissions reduction investments that don’t need it and zero cash to a huge amount more that might. A similar set of problems applies to limiting funding to particular technologies, given efficiency and subsidy requirements will differ by country and investment. (There might, however, be a role for the Bank to help develop and pilot new low-carbon technologies and approaches specific to the challenges of developing countries).

An alternate, and considerably more scalable, approach would be to use the greater lending power of an IBRD capital increase combined with more aggressive use of existing equity to support projects broadly in line with country Nationally Determined Contributions agreed at United Nations Framework Convention on Climate Change (UNFCCC) Conferences. If these loans were predominantly low-transaction-cost policy-based lending, they would be more attractive to clients. While such an approach would not be able to demonstrate additionality, and might back some comparatively cost-ineffective mitigation projects, it would provide considerably more resources toward the $100 billion global climate financing target without the challenge of efficient subsidy allocation. And my colleague Nancy Lee suggests that donor guarantees could help expand financing further by taking the additional exposure off multilateral development bank balance sheets.

The World Bank’s new pandemic fund appears to be facing some of the same issues regarding targeting, with the additional challenge that there is not a measure akin to tons of CO2 equivalent around which even a theoretical market could be delivered. On the plus side, however, the amounts required to simply provide subsidized funds to all lower-income countries to help meet country-level capacities in health security is far more manageable than the investment costs of climate mitigation. Furthermore, the World Bank client countries where the global public good of pandemic preparedness is least supplied are the poorest countries (as opposed to upper-middle-income economies in the case of low CO2 emissions). Most of the grants from the pandemic fund should therefore flow to countries where grants will have their greatest impact, all else equal.  

Still, the lack of progress around efficient subsidy financing of global public goods as a whole is a concern: the idea of global public goods has been around for a half century or more, financing their provision in developing countries has been a concern since at least the 1980s and with climate in particular since the 1990s. And yet we don’t appear to have figured out a replicable model or a robust second-best approach to using subsidies efficiently.

In a world of infinite international grant finance, inefficiency of allocation would be irrelevant. In a world where international grant finance for global public goods was truly additional to existing finance, it would be of lesser concern. But we’re not in those worlds. Figuring how to spend this money well, especially given the high risk that it will be financing that would otherwise flow to IDA (currently providing grants and credits worth about $20 per capita per year to its client countries, with plenty of space to do more), should come before the money starts flowing, not after. If relatively efficient development finance is replaced by considerably inefficient finance supposedly helping achieve global public goods, it is a loss for the planet.

Climate financial intermediary funds (FIFs) represent one of the largest sources of multilateral grant and other concessional finance for climate, including for middle-income countries (MICs). Together, they have received more than $50 billion in cumulative grant funds from donors. They have collectively allocated $48 billion for projects and $2 billion in administrative overhead. 

In a new paper, we look at the structure, size, and performance of the three major climate FIFs: the Global Environment Facility Trust Fund (GEF TF), the Climate Investment Funds (CIF), and the Green Climate Fund (GCF). Together, these funds account for more than 80 percent of FIF financing. At a time when grant and concessional climate finance is scarce, the critical question is whether donor resources are going to projects and countries where they are most needed, most impactful, and most catalytic.

Our analysis reveals significant challenges at the systemic level and differing performance across FIFs. Mitigation finance has generally gone to the countries and sectors with the highest emissions, but country mitigation finance volumes are generally not correlated with the size of country emissions. For adaptation finance, the system does not target the world’s most climate-vulnerable countries. None of them is among the top 10 recipients of FIF adaptation finance.

FIFs provide most of their grant and concessional climate finance to MICs, which receive 84 percent of climate FIF commitments. But LICs receive a larger share of their climate commitments in the form of grants (rather than loans, equity, or guarantees). Most FIF financing is in the form of grants (80 percent), and nearly three-quarters go to public sector recipients.

Figure 1. Climate FIF commitment volumes by instrument and country income group (billion USD)

Note: Only includes data from the CIF, GCF, and GEF; estimates are cumulative.
Source: CIF, GCF, and GEF project databases.
Author’s calculations based on project databases.

FIF funding is not allocated according to consistent criteria measuring results and impact, nor are there consistent results and impact reporting standards. Some FIFs report ex ante impact targets; others do not. There is no uniform ex post reporting standard across FIFs based on a common set of core indicators, making it impossible to compare value for money across FIFs. This makes it hard for donors to assess where best to put their scarce grant resources. The evidence, in fact, does not suggest that donors look closely at FIF performance when deciding where to put their funds. For instance, donor contributions to the GCF have grown most rapidly in recent years, though it has been the weakest performer based on the criteria laid out in this analysis.

Unsurprisingly, financing volumes remain very low compared to needs. Overall, FIF commitments of $4 billion per year remain far below levels needed, especially given calls for more concessional lending terms to incentivize more MIC mitigation investment. This volume is clearly also small in relation to overall World Bank Group climate-related finance of $28 billion in 2021 and combined MDB climate-related finance of $50 billion.

The country composition of the FIF donor base is mostly the same as IDA’s and has not changed very much over the past decade. Emerging donors have not stepped up as major contributors. China, for example, could be a significantly larger contributor to climate FIFs, given its importance as an emerging donor and its stake in global GHG emissions reduction.

Finally, climate FIFs vary widely in administrative costs relative to commitments and relative to project numbers. Some ratios of cumulative administrative expenses relative to commitments range up to 20 percent, while others are in the low single digits.

Making the climate FIFs work better

Clearly this is a system ripe for reform, and in our paper, we lay out a series of options. There is opportunity to capitalize on the MDB reform agenda. We propose consolidating the FIF architecture and merging some of the stronger performers into a single independent climate entity. We think that this could better service recipient countries and implementing agencies, strengthen finance allocation, consolidate administrative expenses, rationalize and simplify fundraising, and combine and scale complementary projects.

But key to getting climate finance right is establishing efficient models for allocating concessional resources and grants—especially if middle-income countries are to obtain a larger share for mitigation. We recommend that FIFs adopt common allocation criteria covering:  

• projected mitigation and adaptation impact
• the scale of impact relative to country goals and challenges
• the country’s need for concessionality
• for mitigation projects, the project’s contribution to global emissions reduction
• for adaptation projects, the country’s global vulnerability ranking

Allocation methodologies should aim to both maximize ex ante mitigation and adaptation impact and maximize impact per dollar committed. This would require agreement across the FIFs and entire MDB sector on a common methodology for projecting and reporting emissions and adaptation impact and for assessing the need for concessional finance.

And finally, donors and FIFs alike should take issues of financial efficiency seriously. There is a clear tradeoff between terms and volumes which will vary across recipient countries, and getting that equilibrium right will require flexibility and new tools. Except for the CIF’s Capital Market Mechanism, the climate FIFs have not sought to innovate their financial model.

There is strong merit to exploring other ways to deploy FIF financing. For instances, FIFs could consider pooling some of their funds to issue guarantees of some share of climate-related MDB portfolios. That would free up MDB capital and allow use of leverage to generate multiples of that additional capital in more climate lending capacity. Such guarantees at the portfolio level are a more efficient way to expand the impact of donor resources than a cash-in/cash-out approach or a transaction-by-transaction approach. And donor contributions can also be used at the portfolio level to make MDB lending more concessional by blending MDB hard loan resources with grants.

Donors and recipient countries alike need an efficient system that allocates finance faster and evaluates value for money more consistently, so that climate financing is put to optimal use in the countries that need it most. To see our analysis and full set of proposals, please read our paper here.