Is an outcomes-based approach or impact bond right for me?
Four dealbreaker questions to ask.
As specialists in the design and implementation of outcomes based contracts, Social Finance is often asked how to decide whether to use an outcomes based approach for a particular intervention or issue area. On this page, we outline four dealbreaker questions that can help service providers, donors, governments, and philanthropies to identify whether outcomes based contracts are appropriate for the social challenge they wish to address.
Since launching the world’s first impact bond in 2010, Social Finance has worked with hundreds of clients and partners to help them to explore whether an outcomes based contract is the right tool to address a specific social issue or need. Outcomes-based approaches – also known as pay for performance or payment by results – can have a range of benefits. At their core, outcomes contracts help to align incentives between governments, donors, service providers and, sometimes, impact investors, to drive better outcomes for vulnerable populations. Impact bonds are a sub-set of outcomes based contract that involve private capital from impact investors to provide upfront funding for the delivery of services, with repayments based on the successful delivery of outcomes.
While these types of approaches can add value across a wide range of issue areas, they are not the right solution for every social problem. There are some cases where grants or service contracts are capable of achieving strong results; and others where the value-add of outcomes based approaches is not sufficient to justify investment in contract design. It is important that government and donor interest in experimenting with new forms of funding does not obscure the primary objective of improving outcomes for the target population.
Our experience has informed four questions to assess whether an outcomes based approach is applicable. These questions can support governments, donors, philanthropies, and service providers to undertake an initial suitability assessment before investing in more detailed programme design.
-
When considering whether an outcomes based approach is suitable for a particular social problem, the first consideration is the definition of success:
Problem: can we clearly articulate the social problem we are trying to solve?
Population: can we clearly identify the population we want to support?
Progress: can good outcome(s) for this population be clearly defined?
Partners: if there is more than one outcome funder, are the outcomes they are seeking compatible? Do service providers, and potential service users, agree on the need to achieve those outcomes over others?
For some social problems, the definition of success is relatively clear cut: for instance, reduced prevalence of a certain disease, or increased employment for at-risk groups. However, this is not always the case. Outcomes based approaches are typically multi-stakeholder initiatives, so aligned objectives are crucial. Social issues with complex, interlinked, or longer-term outcomes may lack stakeholder consensus around what “good” looks like.
For example, an issue such as refugee integration is multi-faceted, with no clear consensus around the problem definition, nor criteria for success. For some, unemployment is the most important problem to solve and sustained employment is the key marker of integration, while for others language learning, mental health, or social bonds with the new community may take priority.
There may equally be differences of opinion over the definition of the priority population. Refugees in the UK typically fall into one of two groups: some have been resettled from an in-country refugee camp by United Nations agencies. These tend to be vulnerable families with complex needs. Others arrive independently in the UK and were granted asylum—these refugees are more frequently single, male and of working age. For various reasons, including practical concerns around the ease of supporting a refugee into work, or a perception of which group has the most significant needs, stakeholders may wish to prioritise one of these two groups.
Those working to prevent homelessness also face questions around which population to support. Some prefer to focus services on individuals who have been living on the streets for a long time who may have the most complex needs. Others may prefer to focus services on those at risk of becoming homeless, like young, single men, where success in moving people into long-term accommodation may be more important.
-
Once the target population and definition of success is clear, it is important to consider the rationale for using an outcomes based approach, as opposed to a grant or a fee-for-service approach.
A rationale might include one or more of the following:
Strengthening incentives: Outcomes based approaches would create stronger incentives to achieve long-term, sustained results.
For instance, Social Finance is working in partnership with Oxfam and local government in Turkana County, northern Kenya, to use outcomes based approaches to drive access to reliable water in this arid region. Funding water access via inputs tends to incentivise a focus on infrastructure, such as the installation of new water pumps and access points; time-bound projects which can be carefully managed, measured and de-risked. However, such contracts do not incentivise long-term maintenance of infrastructure, leading to water points being abandoned if even a minor fault leads to a breakdown. Payment for outcomes, such as reliable access to safe water, could incentivise the creation of more resilient water systems where faults are fixed and infrastructure rehabilitated. It would also be a more cost-effective use of donor funds, when compared to paying for new infrastructure, or humanitarian response when drought hits.
Enabling flexibility: Outcomes based approaches would allow service providers the flexibility to maximise results.
In an outcomes based approach, service providers have considerable financial and operational freedom. They can adapt their operations where necessary, using real-time data to innovate to find the best way of achieving results. The approach’s singular focus on outcomes frees them up from activity-based agreements, which offer little flexibility in the face of changing circumstances. Outcomes based contracts allow providers to focus on solving problems, adapting interventions that are not working and testing new approaches where necessary.
Outcomes based approaches that harness external investment to pre-fund service delivery – known as Social or Development Impact Bonds, or Pay for Success Contract – can enable greater flexibility by transferring the financial risk around whether outcomes are delivered to the investor. This availability of upfront funding can enable a shift towards payments for outcomes rather than outputs. This flexibility is more valuable in some contexts than others; we consider where and why it is most useful in section 3 below.
Ensuring value-for-money: Outcomes based approaches can ensure public money is used effectively.
Tying funding to results ensures that the impact of funds committed by governments, donors or philanthropic foundations is maximised. For example, multiple studies have shown that globally, there tends to be a weak correlation between educational outcomes and government spending. Outcomes based approaches can help to mitigate risks that programmes fails to deliver end-user impact, by ensuring that governments and donors only pay for programmes that deliver results.
In higher income countries, outcomes based approaches have been used to good effect in preventative programs – not least because these are some of the most challenging programmes for governments to fund. Commissioners can face challenges in funding preventive interventions alongside existing services as resources are often fully committed to more urgent remedial services. Preventative services aim to reduce public expenditure on remedial services in the future. One example is recidivism, where sentencing and imprisoning offenders is much costlier than offering preventative services such as skills training, parenting support and mental health counselling. The first Social Impact Bond, launched in 2010 funded preventative services for short-sentence offenders as part of the One Service – an umbrella organisation designed to respond to the complex needs of offenders to help them break the cycle of reoffending. The Impact Bond reduced reoffending of short-sentence offenders by 9%, compared to a national control group. Other preventative Impact Bonds have focused on those on the verge of homelessness, at risk of substance abuse, or on a pathway towards unemployment.
In lower- and middle-income countries, outcomes based approaches are typically used to ensure that committed government and donor spending achieves value-for-money through a tight connection with results.
Fostering collaboration: Outcomes based approaches can incentivise multi-agency joined-up working.
While public sector agencies often have overlapping responsibilities and seek similar outcomes, in reality, funding is often siloed within agencies, leading to fragmented service provision. For instance, recovery from addiction is an issue area which requires interventions relating to health, social services and in some cases housing and employment. However, funding for individuals at risk is rarely joined-up, leading to duplication, misalignment, or, worse, gaps in service provision. Refocusing funding around outcomes rather than interventions can foster collaboration between public bodies and / or donors to address complex social issues.
If none of the considerations above are relevant for a particular social issue or intervention, then an outcomes based approach may not be the appropriate tool. In our work, we often come across examples of projects where a stakeholder’s objectives could be adequately met using other contracting structures. For instance, scaling-up a student loan scheme may be more simply financeable through conventional equity or debt; as would the installation of traditional urban infrastructure like on-grid electricity or piped sewerage networks. However, the addition of outcomes based incentives may ensure adequate service coverage for lower income or excluded groups if additional funding is needed to offset the lower commercial viability of serving such groups through off-grid energy or water access.
Outcomes based approaches offer advantages in terms of stakeholder alignment and operational flexibility, but can also be more complex to design. If the use case is appropriate and the programme has sufficient scale, then this upfront investment is likely to be worth the effort. Higher design costs should be offset by lower performance monitoring costs during programme implementation.
-
It is helpful to consider whether there is sufficient uncertainty around the delivery of outcomes to justify an outcomes based contract. If an intervention is very well evidenced and tightly linked to outcomes delivery, it may be simpler to use a grant or service-contract.
Uncertainty around whether outcomes will be delivered can be driven by a number of things:
Uncertainty of intervention: this is the simplest form of uncertainty, and refers to a scenario where there is some, but not a strong, evidence base for what will drive better outcomes for a particular target group. For instance, the Greater London Authority targeted the London street homeless population through a series of Social Impact Bonds (SIBs) implemented between 2012 and 2016. These outcomes based contracts put homeless individuals at their heart and did not specify which interventions should be used in order to achieve them. This reflected the fact that individual needs to support a sustainable move off the street differed significantly within the target population. Outcomes were structured around sustained tenancy, transition to employment, and reduced usage of emergency health services. Service providers were encouraged to trial new approaches, building up the evidence base for “what works” throughout the implementation period.
Uncertainty of context: applying an intervention that is well-evidenced in one context to another context (geography, target population, or economic climate) means that adaptation of the model is likely to be required. For instance, the Cameroon neonatal health Development Impact Bond (DIB) built on a well-evidenced intervention, kangaroo mMother care, which had been demonstrated to improve neonatal morbidity and mortality rates through continuous skin-to-skin contact between the mother and baby in a number of countries, including Colombia and India. However, kangaroo mother care had not previously been used at scale in Cameroon, and the lead service provider has had to adapt the intervention in real time to fit with hospital protocols, resource constraints and cultural challenges in order to achieve the best outcomes in this new context.
Uncertainty of scale: scaling even well-evidenced interventions in similar contexts comes with inherent uncertainties. As the programme scales, new population groups with varied and different needs will need to be targeted, and operations will need to be adapted to manage the increased complexity that comes with organisational size. For instance, a recent DIB launched in Cambodia involved iDE, a market-facilitating non-profit, stimulating the scale-up of local sanitation entrepreneurs. The intention is to scale access to modern sanitation to rates above 85%, compared to only 29% when iDE started operations in 2012.
We sometimes think about the level of risk around whether outcomes will be achieved through what we call a spectrum of uncertainty.
On one end of the spectrum, uncertainty around the intervention, context, or scale is very low, and providers know exactly what works to achieve results. Where this is the case, the rationale for using an outcomes based approach may be weaker:
First, in very well-evidenced programmes, little innovation and operational adaptation is required to produce outcomes. A scenario like this might occur where there is strong evidence of an effective and replicable intervention model, such as a thoroughly tested vaccine; or where the optimal sequence of activities is already understood, such as in road construction or the provision of loan products to low income populations. Consequently, the opportunities for adaptation and innovation provided by outcomes based approaches may not be required unless the programme is being rolled out at a much larger scale or to a completely new population. Typically, such programmes may be more simply funded through grants or service contracts, equity or debt.
Secondly, when the pathway to achieve outcomes is clear, there is little need to use an outcomes based approach to protect the donor from the risk that they pay for services that don’t deliver outcomes. As described, the most cost-effective and simplest way forward may be for governments or donors to simply specify via traditional fee-for service contracts.
The other end of the uncertainty spectrum is where little is known about how to reliably drive better outcomes for a particular population and uncertainty is high. In this scenario, it is likely that few service providers or, in the case of a Social or Development Impact Bond, social investors, would be willing to accept the level of risk required to contract on an outcomes basis. In these contexts it may be appropriate to trigger payments primarily against more certain output metrics, potentially retaining a small amount to incentivise later stage outcomes.
The optimal point for outcomes based approaches is towards the middle of the uncertainty spectrum when:
We can identify concrete sources of uncertainty which affect the likelihood of achieving the desired outcomes with the target population;
The level of uncertainty is large enough that service providers would value the flexibility provided by an outcomes based approach, and that the outcome funder sees value in paying for results; and
Existing evidence on the achievement of outcomes with the target population is sufficient to persuade service providers or social investors to pay for upfront delivery costs and assume outcomes risk.
-
This is the question that moves us from considering outcomes based approaches more generally to Impact Bonds in particular.
In other results-based approaches, risk is transferred to the service providers – they must finance their activities and will be repaid on achievement of agreed milestones. For example, FCDO’s recent WASH Results programme, targeting improvements to water and sanitation access in Africa and South Asia, worked with larger INGOs such as Oxfam, Plan UK and SNV, who were able to provide their own up-front funding ahead of results-based payments.
It should be noted, however, that many non-profit and profit-with-purpose service providers, particularly smaller, local and more risk-averse providers, may be unable to take on that kind of risk. In such circumstances, government and donors may be forced to specify multiple activity-based payment triggers, solving the cashflow constraint for the provider, but limiting the scope for adaptive delivery of services to maximise impact. An FCDO research study report into supplier access to prefinance in payment by results contracts states:
“The need for pre-financing confers a clear advantage to certain organisations when bidding for PbR. First, organisations with large unrestricted reserves, whether private companies or charities, can often quickly mobilise these funds to cover the working capital needs of PbR contracts. They are also either able to absorb the financial risks of partial-payment linked to not attaining full results or they are not fully accounting for these risks. Second, organisations that can access commercial loans are also at an advantage in terms of their ability to pre-finance PbR. These organisations tend to be private companies with financial track records or significant assets against which they can secure loans.”
Bringing in an upfront funder as a social impact investor within an Impact Bond means that service providers have access to working capital and do not take on the outcome risk themselves (although they might share it with the upfront funder in some circumstances). This enables much greater scope for adaptive service delivery, while enabling the outcomes funder to only pay for results. It also creates space for innovative and / or riskier interventions by transferring delivery risk to the investor, rather than the service provider or outcomes funder.
However, the Impact Bond opportunity needs to be attractive to potential investors. They will be particularly interested in the outcomes risk of the program and the commensurate risk premium they can expect from their investment. Risks they consider will include:
Evidence of previous effectiveness of the proposed intervention;
The operational and financial management capacity of the service provider;
Their assessment of the service provider’s ability to adapt and innovate and, linked to that, their performance management capacity;
Contextual factors out of the program’s control that might substantially affect the delivery of program outcomes – this will include the sensitivity of the proposed outcome evaluation approach to detect outcomes improvement accurately, and the appropriate allocation of political, act of god and other major contextual risks to donors and government.
Most investors willing to take outcomes based risk are driven by a desire to both achieve significant social impact and recoup their funds, often with a premium to compensate them for the outcomes risk they have taken.
Bringing in an upfront funder adds another actor and so complexity to the arrangement. The size of the potential payment by results contract should therefore justify the fixed costs (structuring and management) associated with an impact bond. It may be helpful to reach out to a small number of potential investors to gauge their interest and understand their constraints before progressing to a more detailed design phase.
To re-cap, the questions you need to ask yourself when considering whether an outcomes based approach is appropriate for your needs are:
Do all stakeholders agree upon the social problem and a definition of success?
Is there a clear rationale for using an outcomes based approach? Including: incentives, value for money, flexibility and collaboration
Is there an appropriate amount of uncertainty to make an outcomes based approach a good fit?
Is it possible to involve a private capital to pre-finance service delivery?
Considering an outcomes base approach for your project?
We’re always delighted to discuss opportunities to support an assessment of whether an outcomes based approach could add value in a particular scenario.
Outcomes based approaches in practice: Examples from Latin America
Social Finance’s outcomes contract suitability framework has been designed to support governments and donors to assess the appropriateness of using outcomes based approaches to address a particular social need.
In this piece we reflect on our experience of conducting a suitability assessment for several children’s programmes in Colombia.
In April 2020 Social Finance worked with Henderson y Alberro, our partners in Mexico and Colombia, to assess the suitability of using outcomes based contracting to improve the performance of four public programmes for young people in Colombia. The assessment, funded by the Instituto Colombiano de Bienestar Familiar, the Departamento para la Prosperidad Social and UNICEF, applied our suitability questions to shortlist the four suggested programs before commencing more detailed analysis.
Here we share an overview of findings from two of these suitability assessments, demonstrating how the framework can be used in practice, the process undertaken, and the subsequent benefits observed.
Case study 1: UNIDOS – Improving access to education for children with disabilities.
The first programme we looked at was an initiative seeking to improve access to education for children with disabilities.
1. Do all stakeholders agree on the social problem and a definition of success?
Through interviews conducted on the ground, there was broad consensus around the need to better integrate children with disabilities into the education system, but evidence for what works appeared to be lacking. The rationale for testing an outcomes based contract was to promote innovation and flexibility in how programmes for these children were delivered, as a way of identifying the most effective approaches to improve their outcomes in a school setting. Further to this, clear target outcomes were defined as “improved access and permanence of the population in the education system”. It was reasonably assumed that this objective could be translated into several success metrics suitable for payment-by-results (PbR) contracting which would be attractive to all stakeholders involved.
2. A clear rationale for using an outcomes based approach
Assessing the potential value-add of a PbR approach in achieving better outcomes was central to our thinking. One obstacle to realising this value was the size and degree of dispersion of the proposed target population. The population is estimated at 3,500 families throughout Colombia who have children aged 5–20 with disabilities who do not currently receive specific support to attend school. This is a relatively small population scattered across a big country, which curtails the ability to implement adaptive management practices – thereby limiting the type of learning and innovation that would be conducive to improved results over time.
3. An appropriate level of uncertainty to make an outcomes based approach a good fit
Perhaps more worrying was the high level of uncertainty that still prevailed in the proposed intervention area. Although PbR mechanisms are best suited to contexts where outcomes achievement is not guaranteed, no reliable evidence of an intervention that could achieve the desired outcomes could be identified.
Despite the strong stakeholder alignment observed around the ultimate outcomes desired for the population, it was decided that the level of intervention risk was too significant for a PbR model at this stage. This was driven by the lack of evidence around effective interventions, coupled with a small and geographically disbursed cohort size. It was advised that small grants to facilitate pilot interventions and build the evidence base would be advisable before pursuing an outcomes based approach.
Case study 2: Primera Infancia – Early childhood development services.
The second programme reviewed focuses on Early Childhood Development (ECD). Across Colombia, childhood development centres (CDIs) operate for children between the ages of 6 months and 5 years, providing basic early education and developmental support before children reach primary school. The suitability framework was used to investigate the potential of PbR approaches to improve the quality and cost-effectiveness of the services provided within the CDIs.
1. All key stakeholders agree upon a definition of success
At the outset of our study, it was unclear whether the ambition was to enhance the quality and value-for-money of the CDIs themselves, or to improve the child-centred outcomes resulting from the activities of those CDIs. Through a process of exploration with key stakeholders, it was ultimately agreed that the goal ought to be an improvement in early childhood outcomes, since there was no evidence linking specific quality metrics within the CDIs with child outcomes – so metrics around the quality of the CDIs might not be indicative of improved outcomes for children. Establishing agreement around outcomes allowed us to start thinking in terms of concrete interventions.
2. A clear rationale for using an outcomes based approach
The lack of clarity around which CDI interventions and practices were most effective in delivering better outcomes for children presented a strong justification for the test-and-learn approach inherent in PbR mechanisms. Moreover, the diversity of interventions already being used by the CDIs pointed to the viability of, and appetite for, experimentation in the market. Finally, unlike in the previous example, the size of the target population, both in terms of the number of children using ECD services and of the number of CDIs delivering them (the likely intervention unit), was very significant, offering a unique opportunity to leverage adaptive management practices to improve outcomes over time.
3. An appropriate level of uncertainty to make an outcomes based approach a good fit
There were a host of unknowns associated with the ECD programmes being operated across the CDIs. Interventions varied, as did the quality of data and measurement. However, we were able to identify effective, evidence-based interventions for the improvement of early childhood outcomes. Outcomes based contracting can deliver significant value in contexts where better outcomes are sometimes, but not always, delivered.
4. Potential to involve private capital to pre-finance service delivery
A challenge in applying PbR in ECD is that the benefits of strong interventions materialise over a much longer time horizon than is typical for outcomes based contracts. Outcomes such as improved reading or writing may only be measured after 2–3 years and may not result in measurable academic success until much later. This has sometimes limited investor interest in outcomes based contracts for ECD in other countries. However, CDIs in Colombia have historically been funded by a combination of public and private sector actors. The current presence of private philanthropy and investors in the sector is promising for implementing a PbR initiative. We concluded that a PbR scheme, with robust proxy measures for longer-term outcomes, would likely be able to attract both private and public sector actors.