What needs to be evaluated, and how, if the climate emergency is treated as an immediate and present danger to environmental, social and economic well-being?
If the ‘climate emergency’ is to be taken seriously, there is a pressing urgency to make progress. But the fact remains that progress also needs to be measured and demonstrated – and there are many important lessons to learn. Doing such evaluation takes time, of course, so there is a tension at the heart of the climate emergency: how to move more quickly, but also be reasonably sure that the moves are the right ones?
Energy Evaluation Europe conference
Energy Evaluation Europe (EEE) is a two-yearly conference, which took place as a virtual event for the first time in March 2021. EEE is a gathering of public and private sector evaluators, evaluation funders, policy makers and researchers from across Europe and beyond. This year’s conference theme was ‘Accelerating the energy transition for all – evaluation’s role in effective policy making.’ As Hans Bruyninckx, Executive Director of the European Environment Agency, asked in his keynote presentation on Day 1: “can you go on with evaluation as usual in ‘business as unusual’?”
On Day 2 we held a debate on ‘Evaluation in a time of emergency’, chaired by Tina Fawcett (Oxford University/CREDS), with panellists Lelde Kiela Vilumsone (European Commission, DG Energy), Harsh Pershad (Innovate UK) and Gavin Killip (Oxford University/CREDS). We wanted to prompt discussion about what would need to be evaluated, and how, if the climate emergency was treated as an immediate and present danger to environmental, social and economic well-being. What role should evaluators play in helping to identify and deliver rapid and effective responses to the climate emergency?
Current evaluation practices
Lelde Kiela Vilumsone started the panel with a description of current evaluation practices and processes within the EU, and the challenges in undertaking high quality evaluation. She noted that evaluation of energy efficiency policies is already very complex, with problems of uncertain baselines, interaction between policies and rebound effects, amongst others. In the context of more ambitious climate targets, evaluation will become even more important to allow shaping more targeted and effective policies. Sharing best practices between Member States and building on existing knowledge should also continue. Lelde Kiela Vilumsone suggested that digital data and tools could help deliver more robust and timely evaluation.
How will we get to net zero?
Harsh Pershad noted that ‘net zero’ CO2 emissions is one of the largest of many huge challenges facing societies. Citizens, governments and businesses don’t yet know or agree on how they will get to net zero and will need to ensure other priorities are also met (e.g. costs, competitiveness, and service levels). More actors (public, private) will need to align, invest, and innovate together to drive systems changes. Whereas historically evaluation often considered technology readiness, units installed or energy flows, against resources invested, for systems change it is important to create cultures and processes to support multi-dimensional evaluation. For evaluators, challenges will include partnering with teams in different organisations to manage diverse dataflows across complex and interdependent projects, whilst keeping stakeholders in different organisations appropriately informed, engaged and motivated to exchange the most insightful data in a timely manner. Evaluators that succeed will fuel the essential virtuous circles of feedback, improvement and stronger stakeholder relationships in multiple systems that will be needed to drive net zero.
Innovation and evaluation
Gavin Killip presented insights from the acceleration of innovation and evaluation which has taken place as part of the development of Covid-19 vaccines. New vaccines have been developed more than ten times more quickly than in normal circumstances, so what can energy/climate policy learn from that rapid success? Clearly there are differences of substance and context between the climate and Covid emergencies, but nevertheless there are still useful lessons and insights. The political willingness to take risks (the ‘whatever it takes’ approach) was matched by an extraordinary number of trial volunteers coming forward. Those things were then mobilised by established organisations with the expertise and organising capabilities to deliver speed and quality simultaneously.
In response to questions from the audience, the debate touched on making best use of new sources of energy data, e.g. from smart meters, and new techniques such as creating digital twins of energy systems. In addition, increased knowledge exchange, through existing forums such as Concerted Action (for EU Member States) and Energy Evaluation Europe, is important. Skills development is required within the evaluation community, including new skills around engaging with people and increasing awareness.
There will need to be more flexibility in evaluation and identifying success criteria. Some of the important aims of innovation projects – like influencing the culture – are inherently very difficult to evaluate. New methods are needed to enable this and to recognise multiple benefits. While delivering energy or carbon savings is more important than ever, rapid and ambitious projects are more likely to see a higher failure rate. Embracing the risk of failure – and learning from it quickly to inform future projects – will be part of responding to the emergency.
Conclusion
Ultimately the panel felt evaluators need to develop skills and methods to help meet the challenge of the climate emergency, but this can only happen in collaboration with policy makers, funders and those delivering projects. It is not a change they alone can lead. Further thoughts on this topic can be found in a commentary paper by Sam Hampton and colleagues.
EEE papers and extended abstracts are available now and many of the plenary and parallel session recordings will be made available between April and July, with the first already on YouTube. To learn more, you can subscribe to the Energy Evaluation newsletter.
Banner photo credit: Science in HD on Unsplash
