-
-
- Simon Clarke MP delivered a short speech for the ETF’s Policy that Works conference to support the most efficient spend of taxpayer money through honest reflections on whether programmes are effective.
-
By Simon Clarke
The Policy that Works conference is a fantastic innovation and I’m certainly a great believer in the crucial importance of evaluation alongside an approach to government with the courage to try new things and approaches.
The government spends roughly a trillion pounds a year – that’s obviously a colossal amount of money. And we need to ensure it’s spent wisely on the right things with a constant focus on delivering value for money for the taxpayer. And there are always arguments about whether we spend too much or too little on health or education or defence or foreign aid or any of the other issues that we cover.
But, of course, there’s another question that cannot be asked enough: does it work and is this effective? We need to be asking ourselves those questions through robust evaluation of our policies and our programmes at all times.
Now, that means accepting bravely, in a way that’s sometimes hard for us to do, that not everything will work on every occasion. That can be very, very difficult but robust evaluation will and should allow us to see that some interventions aren’t as hopeful and as effective as hoped for, including some that we really liked, and were excited about. That can be frustrating or inconvenient, even embarrassing for politicians, but it is the right thing to do.
Now, we can all smile at apocryphal examples, including the Cobra Effect which I’ve been learning about, so this was named after rewards offered by the British Empire in India for the capture and killing of cobras in Delhi. Unfortunately, I think we can see the flaw here – enterprising local farmers soon took to breeding them so they could kill them and claim the reward. Realising it wasn’t working, officials stopped the programme, leading to the farmers releasing the snakes and making the problem even worse.
This is a classic example of the intended benefits and actual results of programmes sometimes being more settled than we realise, and it can be methodologically challenging to anticipate these things. But we need to always try to answer what works. For whom, when and at what cost? There are many examples of things which work better than the Cobra effect, and there are brilliant examples of evaluation being used right across the Civil Service through the excellent work of our policymakers, analytical professions and with support from bodies, such as the What Works Centres.
Now, you will hear about some of these excellent examples throughout this conference, and we need to do more of this. We need to put evaluation at the centre of enabling and creating a more innovative approach to policy making. This is where we try something completely different. Think about the supporting families programme or auto-enrolment on pensions or the introduction of e-cigarettes or in this building, most recently, the furlough scheme. These programmes or interventions went beyond a marginal change. They were intended to be radical breaks in practice. Some of them have been an amazing success.
Auto-enrolment in pensions, introduced ten years ago, has led to more than ten million more people saving for their pensions and has transformed the pensions landscape.
If I were asked to summarise my main points to you today, the first would be that ministers, including myself, are deeply committed to evaluation. I really want and welcome advice from officials to back up reasoning with evidence of what has worked before and is working already. I know that one of the most common arguments against robust testing evaluation can be that ministers don’t really want to know whether their precious ideas and programmes work.
In true “yes minister” style, Sir Humphrey might expect it is better to spend a few billion extra on something that doesn’t work, but at least he can be seen to be acting. Then to find out, of course, what they did doesn’t work, and to direct the money to something that does. I would very much emphasise the message that we do indeed want to know as it is our absolute fiduciary responsibility to safeguard public money. It is absolutely imperative to us that the policies that we’re delivering with that resource are effective.
Now, the second point we’re building, the Treasury needs to and will practise what we preach. It is very hard to build randomised control trials into every policy. We’re not going to try adding a penny on beer in Bradford, but not in Birmingham, you’ll be relieved to hear, to check the impact on A&E on Friday nights.
But that doesn’t mean that there aren’t lots of ways of testing and evaluating the impact of what we do. We can run and support robust pilots. We can use discontinuity designs where we use data around the transition point or cut-offs to estimate the impact between those just in and just outside a programme. We can run online experiments or trials to test how people or businesses might respond to a given change. Robust evaluation needs to be at the central decision-making across government and it will be.
Third, I want to encourage you to be daring. Part and parcel of evaluation experimentation is imagination, creativity and innovation. I think those are all things which I know the civil service wants to deliver. We are all doing the jobs that we do because we want to make people’s lives better and that means that we don’t always just do the same thing as we’ve always done before.
Is there a different or a better way? And that doesn’t mean trying out a brand new idea every time on 60 million people, but it does mean making room for active exploration and testing and promising ideas. So please do be daring and be robust. Let’s dare to try out the best ideas that we have and dare to fail as well. Crucially, dare to fail but to fail fast by understanding what it is that’s going wrong, seeing if something can be improved with a programme.
On the other hand, let’s double down on the winners as well. Let’s scale these projects. Let’s roll them out quicker and faster so we benefit more people and we do get things spot on.
I’ll finish by thanking you today. This is only the beginning of a crucial and ongoing conversation about how we can further build evaluation into our policy-making, learn what works and what doesn’t, and ultimately make better decisions for the future.
So, please get involved. Take this conversation back to your teams and thank you above all for all the wonderful work you do in your departments.
