Impact at scale is the new mantra – whether it is for the achievement of the Sustainable Development Goals, for the implementation of the Paris Climate Agreement, or for realizing the goals of the Sendai Framework on disaster risk reduction. Achieving impact at scale, however, remains challenging and is the subject of much discussion. Evaluation plays a key role in effective scaling, but little attention has been paid in the evaluation literature and practice to explore how scaling processes should be evaluated. This two-part blog series by 3ie Senior Research Fellow Johannes F. Linn reviews key aspects of the scaling process and considers how evaluation practices can be adapted to support scaling.
Scaling considered through three different prisms
The first perspective is to consider how a successful innovation (e.g., new climate-smart agricultural practice) can be scaled up through effective diffusion of production, marketing and adoption. This generally involves a sequence of stages commonly referred to as a ‘scaling pathway’ from (a) ideation to (b) research and development to (c) proof of concept to (d) transition to scale to (e) scaling and finally to (f) operation at sustainable scale.
Previous work on scaling innovation shows that for successful scaling it helps not only to have a good understanding of what the innovation is and whether it works, but also to have a clear vision of scaled-up impact to be achieved. The question then is: what are the enabling factors that support the scaling process and pathway from innovation to impact at scale? Among such factors are ‘drivers’ that push the scaling process forward and ‘barriers’ that hinder the scaling process along the pathway (or ‘spaces’ that must be created to allow initiatives to grow). Drivers include champions for the scaling process, incentives (e.g., subsidies), and market or community demand. Barriers that must be removed or avoided include fiscal and financial constraints, institutional capacity limitations, policy and regulatory obstacles, environmental constraints (e.g., land or water availability for agricultural interventions), and the lack of partners.
The way enabling factors affect the scaling process often changes as you move along the scaling pathway, so frequent adjustment will be required. In fact, while often presented as a linear process, effective scaling is iterative. Evidence must be gathered along the scaling pathway to check whether the innovation works as intended at different stages and how enabling conditions affect the scaling process — with appropriate changes needed across all aspects of the scaling process, consistent with the premise of the Problem Driven Iterative Adaptation (PDIA) approach. For this reason, monitoring, evaluation and learning (ME&L) play a key role in effective scaling in order to assess not only whether the desired impact is achieved along the pathway, but also how the various enabling or impeding factors (drivers and barriers) impact the process.
A second perspective is to view scaling as a multi-year, multi-project approach. Most development work is organized in the form of time-bound projects. All too often, development agencies design and implement projects as one-time activities with insufficient consideration of how their impacts can be sustained when successful and how they can be replicated and scaled up.
A key finding of the scaling-up experience is that successful scaling usually takes many years and requires sustained engagement by various actors supporting the process through multiple project cycles. One-off projects will generally not be enough to sustain the scaling process. So, projects should be considered as interventions along a scaling pathway as shown in Figure 1.
A traditional one-off project (Project 1) represents only the first step along the potential scaling pathway. For it to have an ultimate impact at the desired scale (say, at a national level), not only must it have sustained impact (the solid green arrow, rather than the red dotted arrow), but it must also be followed up seamlessly with subsequent projects or interventions that continue the scaling up process. Successful scaling up through projects requires planning during the life of Project 1 (and during Project 2) for what will happen when the projects finish. It needs to consider what enabling conditions have to be put into place to allow the follow-up project to proceed without a break. This includes, very importantly, who will be the actors to take on Project 2 (or 3), how it will be funded, what are the policy, institutional and political conditions required, and what partnerships need to be mobilized. This multi-project perspective can be referred to as a “programmatic” approach, with the program conceived of as a sequence of projects from the beginning.
Figure 1. Scaling up with successive projects
Source: Cooley and Linn (2014)
Again, ME&L play a key role during project implementation to assure the project impact is as intended, that the enabling conditions are effectively put in place and necessary adjustments are made during project execution to assure the scaling pathway is successfully implemented.
A third perspective is to view impact at scale through the prism of system change. This approach to achieve impact at scale typically considers the entire set or a subset of enabling factors – i.e., the system – and asks what changes are needed to provide the appropriate opportunities for potential individual innovations and actors to help achieve the scale goal of developmental impact.
As noted earlier, areas for system change include fiscal and financial policies, institutional development and capacity building, policy and regulatory reform, measures to support environmental protection, and actions to facilitate partnerships (e.g., public-private partnerships). At a country level, these system changes generally require action at the level of national government. They can focus on comprehensive reform changes or on specific sectoral and thematic areas (e.g., agricultural sector reform, including trade tariffs or quotas, subsidies and taxes, market regulation, public investment planning, extension services, etc.).
Certain funders (e.g., large multilateral development banks) support system change at the national or subnational level. In these cases, instead of financing individual innovations or investment projects they support policy and institutional reform, usually with budget support or development policy financing.
As for scaling innovations or project interventions, system change takes time and persistence, rather than one-off action. New policies have to be implemented with the passage of appropriate laws and regulations, which, in turn, have to be administered effectively on the ground. Therefore, system change also requires ME&L to ensure that the reforms are implemented systematically and have the intended effects, as there can well be unforeseen and unintended consequences that require correction in the reform path.
The next piece of this two-part blog series expands on the role of ME&L during project implementation to help understand how evidence can inform the scaling process.
Comments
David, you're absolutely right -- market, community and/or political demand (and not just perceived need need) is a critical driver for an innovation to scale; and institutional capacity (space) to implement the scaling pathways is one of the important enabling factors that has to be present or created. While these factors are noted in passing in the blog, they are indeed worth stressing as important parts of the theory of change underpinning the envisioned scaling pathway.
These are three good models for understanding the scaling process. Ultimately though, to achieve impact at scale we need to look for the institutional and market forces that will sustain the impact without further "project" oriented work, I think. Having a theory of how this is likely to happen is important from the outset so we are making project decisions that support the goal of sustained impact post project intervention.