How to monitor and record impact

This guidance note is the fourth in our series on the journey from research to impact.

1. The research to impact journey: an overview
2. How to promote research
3. How to undertake knowledge exchange
4. How to monitor and record impact

What is research impact? A recap…

Impact is the demonstrable contribution that excellent research makes to scientific advances, society and the economyOpens in a new tab (UKRI). Research impacts are the long-term result of an intervention / series of interventions. Over time, the aim is that the outcomes achieved through knowledge exchange and stakeholder engagement make a measurable difference – for example, in policy, legislation, procedures or mindsets – and lead to the ultimate goal of reducing energy demand, enabling flexible demand and use of decarbonised energy (‘impact’, in the research impact pathway). Tracing impact improves the accountability and quality of your research, as well as maximising its benefits and, potentially, enhancing the reputations of those involved.

Monitoring impact

Regular reviews of research projects are recommended to keep track of whether your project is delivering impact. It is helpful to review what is being delivered, to whom and what use they have made of it. It is also useful to consider what impact was envisaged compared to what has been delivered, where additional benefit has been secured and what has been most successful – or, in other words – what are you most proud of?

Why is monitoring impact important?

• It strengthens the evidence base underpinning your own career as you can mention impacts in your CV and in proposal writing.
• Researchfish and the REF are both based on impact.
• Monitoring your impact allows you to understand your progress. If you find that you have not had the impact you were expecting or hoping for, it gives you the opportunity to iterate and develop new approaches.
• In the end, making a difference is what most of us are here for. Sometimes research can feel a long way from the real world, so monitoring progress helps to keep you motivated.

How to monitor and record impact

The journey from research to impact is iterative and can be complex. Keeping track of it and seeing how connections evolve is not a trivial task.  Evidence of impact can be elusive, partial and uncertain, and is usually a result of months or years of work by several interested parties, so keeping a record is important.

Examples of activities that can be recorded and might lead to impact – Researchfish categories

• Publications: Record any publications, for example, journal articles, conference proceedings, reports, policy briefs, book, guide, other.
• Collaborations and partnerships
• Further funding: Any additional funding & details of funding body and process, for example, research grant, fellowship, co-funding, capital, travel).
• Engagement activities: Details of activities that have engaged audiences, for example, working group, expert panel, talk, magazine, event, open day, media interaction, blog, social media, broadcast).
• Influence on policy: Details of activities that have influenced policy audiences, for example, letter to Parliament, training of policymakers, citation in guidance/policy docs, evidence to government including consultations.
• Influence on business: Details of activities that have influenced business, for example, citation in working procedures, revisions to guidance docs, citation in industry report, article in trade press, talk at trade event.
• Research tools and methods
• Research databases and models: Only list the ‘new’ elements of any models, processes, data, for example, data analysis technique, handling, algorithm).
• Intellectual property and licensing: For example, copyright, patent application, trademark, open source.
• Artistic and creative products: For example, image, artwork, creative writing, music score, animation, exhibition, performance.
• Software and technical products: Any non-IP products that are public or do not require protection, for example, software, web application, improved technology.
• Spin-outs
• Awards and recognition: For example, research prize, honorary membership, editor of journal, national honour.
• Use of facilities and resources: For example, databases from outside of CREDS, shared facilities.
• Other: Anything not already covered above.

Remember that many of the most valuable impacts are hard to measure and so it is worth thinking about indicators of change in the areas you care about. How, for example, would you track a cultural change or monitor a shift in attitudes? There are often creative proxies and tools for such monitoring, or you may create your own.

Looking for impact

Impacts are dislocated from inputs in both time and space and so it can be difficult to attribute direct causality between them. This also makes them difficult to track, especially if your funding changes and you move on to something different. Nevertheless, there are some things that will make them easier to find and here are some ideas:

• Keep track of your audiences. Feedback from your audience can come in multiple forms, from an email or feedback form, at the ‘easier to track’ end of the scale, to a changed business model or policy at the ‘harder to track’.
• When looking for impact it’s important to remember the different types of impact you might have, ranging from changing the course of an important conversation during an engagement event with a stakeholder, through making a methodological contribution, to having the application of your results or recommendations from your research be taken seriously. You might also produce data or a model that is used repeatedly in academia or to underpin a particular policy perspective.
• It is also worth being aware of the various mediums where impact might be logged, from obvious ones like media mentions, likes or citations, to less obvious but more profound ones like shifting a disciplinary boundary or being instrumental in initiating a new funding call.

Recording outputs, outcomes and impact

The CREDS impact monitoring spreadsheet, xlsx is a useful tool to record outcomes from an intervention and assess whether and how impact might be achieved. The spreadsheet allows you to add in activities undertaken (promotion activities), what outcomes were achieved as a result of that activity (e.g. interview for article, meeting arranged) which can then be used as a prompt to undertake further follow on engagement (knowledge exchange) to drive impact. Whatever system you use, organising the evidence of your impact safely and comprehensively as you go along will pay dividends later.

Telling the story of your impact

An effective way of of your impact, is through case studies. During the CREDS mid-term review, we were asked to demonstrate our impact through case studies and EPSRC provided some helpful guidelines:

• Catch the attention: Come up with a snappy title. Write three bullets summarising the main points.
• Be brief: Write a 150-200-word summary abstract that tells the whole story. Use another 100 words to provide more information, so a total length of 300-350 words.
• Be comprehensible: It should be well structured with clear headings, at appropriate levels to aid comprehension. Graphics and pictures can be really helpful, and logos can be included if appropriate but may not be necessary. Avoid jargon and acronyms and do not assume prior knowledge.
• Provide more information: If possible, point to next steps and provide a link to more information. The members of the project team should be named and contact details provided. References can be included if necessary and you should be clear how others should reference the case study, if they want to refer to it (which will make it easier to cite and may improve its impact and dissemination)

Guidance on REF case studiesOpens in a new tab might also be helpful.

Science storytelling: workshop with Josh Ettinger

Transcript of science storytelling video

Summary

Impact is long-term, iterative and cyclical, based on mutual benefit and on-going relationships. It is a journey from evidence, promotion, stakeholder engagement and knowledge exchange, to impact.

Banner photo credit: Gyula Gyukli on Adobe Stock