How robust are your model findings? Samuele Lo Piano explores CREDS’ energy modelling auditing project.
Have you been wondering how robust the findings of your model are? Whether they represent and address the actual issues of the problem owners or just reflect the bias of a particular modeller or modelling school? How potentially detached from reality can the model be?
These are questions that researchers, and civil servants tackled in the CREDS energy modelling auditing project. The activities analysed revolved around the simulation of residential energy modelling demand, building energy demand (also in developing countries), and scenarios for the uptake of heat pumps in the UK to 2050.
Analytical lenses were firstly defined to assist in the scrutiny of modelling activities. Theoretical content and hands-on sessions were delivered on the NUSAP scheme for the communication and management of uncertaintyOpens in a new tab, as well as on sensitivity auditingOpens in a new tab for the scrutiny of policy-relevant modelling studies. Several analytical layers were embraced. One was the global uncertainty and sensitivity analysis that tests the uncertainty and robustness of models when the uncertainty space of parameters’ assumptions is properly explored.
Another concern was the appraisal of the epistemic background and framing of the modelling activities developed, as well as the identification of the potential winners and losers upon the implementation of policies informed by the modelling activity.
In other sessions of the project, the participants made use of these tools on their own modelling projects, analysing their own and each other’s models. The analytical perspective was then widened by involving colleagues with a different background and interest in other aspects of energy demand, to fully appraise the framing of the model. For instance, participants from the domain of social sciences were interested to highlight how energy demand and its temporality came about in the modelling activities. With the participations of scientific officers of the Joint Research Centre of the European Commission and the European Environment Agency, the reflections moved to the actual capability of models to capture the actual political problems at play, and the interplay between different political measures. One issue was how to integrate variable patterns of generation from renewable energy with the observed temporality of residential energy demand. Another issue was the fairness-sustainability dichotomy of applying a price cap on natural gas – potentially helping vulnerable customers, but harming the environment. Finally, the participants discussed how the modelling activities could be taken as representative, given the plurality of lifestyles in the European Union.
Participants appeared to appreciate the importance of looking at one own’s model with a critical eye, and had learned new strategies to do so. We hope that this will lead to the replication of such initiatives in future research projects and consortia.
Banner photo credit: Gyula Gyukli on Adobe Stock
