Outcomes Based Accountability: A refresher
This blog post is the first in a series which will consider Outcomes Based Accountability, or ‘OBA’, in the context of the draft Programme for Government Outcomes Framework. Over the series we will detail the main concepts of the OBA system, and consider the potential and challenges of applying these concepts in the Executive’s Programme for Government.
What is OBA?
OBA is a trademarked system developed by Mark Friedman, and most fully expressed in his 2005 book Trying Hard Is Not Good Enough. The system was originally developed under the title Results Based Accountability or ‘RBA’: the two titles refer to the same system. It has been employed in many national and sub-national contexts around the world, including in Scotland and Wales.
OBA seeks to define agreed outcomes – which are plain English ‘conditions of wellbeing’ for a given population. All work is then driven towards progressing these outcomes. The approach works on two distinct levels: the higher and broader level of population accountability, and the lower and more specific level of performance accountability (Figure 1).
OBA in Northern Ireland
The Executive increased the use of outcome and impact-based commissioning, including OBA, particularly when funding third sector organisations since the early 2010s.
In 2014, the Executive asked the Organisation for Economic Co-operation and Development (OECD) to complete an assessment of its public reform agenda. The OECD completed this report in 2016, and made multiple recommendations, including implementation of a multi-year outcomes based Programme for Government.
The Executive’s next Draft Programme for Government Framework was published in May 2016 utilising the OBA framework, with 14 outcomes for the population of Northern Ireland. The OBA approach was retained in the draft Programme for Government published in October 2016.
One month after consultation closed on the draft Programme for Government, in January 2017 the Executive fell and the Assembly no longer fully functioned, during a political hiatus lasting for three years. During this time, the civil service maintained the OBA approach in the Outcomes Delivery Plan, which reflected the Programme for Government at the point of the Executive’s collapse.
In the New Decade, New Approach political agreement of January 2020, parties agreed a two-stage approach: first, developing a Programme for Government which would continue to use OBA, and second, agreeing a Priorities Plan of actions for the remainder of the 2019/20 financial year. The COVID-19 pandemic and Executive response delayed development of the PfG, with a draft PfG Outcomes Framework only launched for consultation on 25th January 2021.
Population accountability
Population accountability is summarised by the Northern Ireland Audit Office as:
[. . .] a system or process for holding people in a geographic area responsible for the well-being of the total population or a defined sub population. At a regional/national level, these are the outcomes or the conditions of wellbeing that we want for our citizens and communities, such as a safe neighbourhood or a clean environment. These outcomes are population outcomes as they refer to whole populations of a city, region or country.
The population accountability process begins with the organisations responsible for delivery asking and answering seven key questions. These questions define the desired outcomes, and how they can be measured and delivered:
- What is the outcome we want for the people in our community?
- How would this outcome be experienced if we could see it?
- How can we measure these conditions?
- How are we doing on the most important measures?
- Who are the partners that have a role to play?
- What works in order to do better, including no-cost and low-cost ideas?
- What do we propose to do?
The outcomes agreed are intended to be broad and aspirational statements, which require sustained action and cooperation across multiple stakeholders to be progressed.
Progress towards outcomes is measured by developing and monitoring indicators. An outcome typically has multiple indicators, providing objective and reliable data which demonstrate progress. For example, Outcome 2 in the 2016 draft Programme for Government is ‘We live and work sustainably – protecting the environment’. One indicator for this outcome is the percentage of household waste that is reused, recycled or composted.
As in Figure 2, the aim is to ‘turn the curve’ on the chosen indicators, by improving performance compared to what would have happened otherwise.
Performance accountability
Performance accountability is the lower level of OBA. Where population accountability focuses on defining and measuring progress for the whole of Northern Ireland, performance accountability does this for an individual client group: for example, all students at the College of Agriculture, Food & Rural Enterprise (CAFRE).
Performance accountability begins with the organisations responsible for delivery asking and answering seven key questions. These establish the client group, how positive impacts for this group can be measured, and how these impacts can be delivered:
- Who are our customers, clients, people we serve?
- How can we measure if our customers/clients are better off?
- How can we measure if we are delivering service well?
- How are we doing on the most important of these measures? Where have we been; where are we headed?
- Who are the partners who have a potential role to play in doing better?
- What works, what could work to do better than baseline?
- What do we propose to do?
After interventions are proposed by the responsible organisations, performance accountability assesses the impact of the interventions by asking three questions:
- How much did we do? (Quantity of work)
- How well did we do it? (Quality of work)
- Is anyone better off? (Impact of work)
Each intervention develops a ‘report card’ (see Figure 3) of agreed measures, which details its work and impact by answering these questions. A CAFRE initiative’s report card could include the number of participants (‘How Much’); the percentage of participants satisfied with their experience at CAFRE (‘How Well’); and the number and percentage of participants gaining qualifications, and/or employment (‘Better Off’).
How do population accountability and performance accountability interact?
Where OBA is adopted, there should first be agreed outcomes and indicators at the population accountability level, which define the long-term vision for the population.
Individual Departments, agencies and interventions should then employ performance accountability to demonstrate progress at that lower level. The work of all interventions, at performance accountability level, should clearly contribute to one or more population-level outcomes.
Given this, it is important to understand the distinction between the population and performance accountability levels of OBA. Performance accountability is focused on individual agencies and interventions, with more immediate and identifiable impacts and beneficiaries. At the population level, however, outcomes are so broad and aspirational that they involve a wide range of stakeholders and influences, and can only be progressed in the longer term.
It is therefore possible for performance level interventions to achieve positive impacts, without necessarily succeeding (alone or together) in progressing a population-level outcome.
For example, Invest NI could succeed at the performance level of OBA by delivering new jobs through small-scale enterprise projects, but the impact of this could then be outweighed by an international recession, which prevents progress at the population level. Successes at the performance accountability level do not guarantee progress at the level of population accountability.
Conclusion
As the opening post in this series, this blog article has focused on the key concepts and structure of OBA. Future posts will consider population accountability and performance accountability in more detail, with a particular focus on their practical value to the Assembly and Executive.
