When the amount of information reported about a study is just right, readers have a better understanding of how results were reached.
Good reporting of scientific research can be a bit of a “Goldilocks” issue. When the amount of information reported about a study is just right, readers can understand what was done and come to an informed judgement on its validity. Other researchers can also seek to reproduce it in full or in part in their own work, further testing the validity.
But knowing how much information to report is not always straightforward. Sharing too little can lead to important flaws and biases being overlooked. Too much, and research reports become impenetrable tomes in which the important details are lost in a morass of operational waffle.
Energy demand researchers (ourselves included) are not immune to this challenge. It’s therefore surprising that reporting guidelines are not used more widely in our field.
A reporting guideline is a list of items which it has been agreed are important to report for different kinds of study. They are usually developed through processes of expert deliberation and consensus. By following a guideline, researchers can make their lives easier by taking some (but not all) of judgement out of how to structure and present their research reports – and avoid tricky peer review feedback too.
But use of reporting guidelines is virtually unheard of in energy demand research. This may be because, unlike in more traditionally disciplinary subject areas, energy studies often do not conform to neat “types” for which a suitable guideline can easily be selected. We argue that this should not prevent critical use of the most relevant guidelines – following them as far as possible, while omitting irrelevant detail and including other key points. Importantly, we think it is better to omit or include details in a conscious, considered way than not consider them at all, and referring to a guideline is helpful in this.
Low use and awareness is likely propagated because students are not taught about the existence of reporting guidelines. But there is growing recognition that this needs to change.
If you are an energy demand researcher and want to know more about reporting guidelines, a good place to start is the Equator NetworkOpens in a new tab While focused on health research, it brings together details on a whole range of guidelines – many of which are also applicable to energy studies. To make things easier, we’ve pulled out a handful of guidelines that we think are likely to have especially broad relevance to energy.
- Evidence review. The PRISMAOpens in a new tab guidelines provide a 27-item checklist of points to report, including search strategy, study eligibility, and risk of bias assessment. If you are conducting a realist review then the RAMSESOpens in a new tab guidelines will be more appropriate.
- Observational studies (such as building monitoring). Here, the STROBEOpens in a new tab guidelines provide useful direction on important factors to report as does its extension, the RECORDOpens in a new tab
- Randomised control trial. Detailed reporting is perhaps especially important here, because experimental work often has reproducibility as a more central aim compared to other research designs. The CONSORTOpens in a new tab guidelines provide a detailed checklist to follow. As with other guidelines, not all points will necessarily be applicable to your study, but it is better to consider and dismiss than ignore entirely.
- Qualitative studies. The COREQOpens in a new tab guidelines were developed to facilitate reporting of interview and focus group studies. Like all the guidelines, these are perhaps especially useful for researchers who are new to the research approach and less sure about which details to report.
- Computer modelling and simulation. A range of guidelines exist here and we have not yet identified a preferred option to recommend. You can find these by searching online. An example would be the STRESSOpens in a new tab
| Field | No. | Description |
|---|---|---|
| Title | ||
| Title | 1 | Identify the report as a systematic review |
| Abstract | ||
| Abstract | 2 | See the PRISMA 2020 for Abstracts checklist |
| Introduction | ||
| Rationale | 3 | Describe the rationale for the review in the context of existing knowledge |
| Objectives | 4 | Provide an explicit statement of the objective(s) or question(s) the review addresses |
| Methods | ||
| Eligibility criteria | 5 | Specfiy the inclusion and exclusion criteria for the review and how studies were grouped for the syntheses |
| Information sources | 6 | Specify all databases, registers, websites, organisations, reference lists and other sources searched or consulted to identify studies. Specify the date when each source was last searched or consulted |
| Search strategy | 7 | Present the full search strategies for all databases, registers and websites, including any filters and limits used |
| Selection process | 8 | Specify the methods used to decide whether a study met the inclusion criteria of the review, including how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process |
| Data collection process | 9 | Specify the methods used to collect data from reports, including how many reviewers collected data from each report, whether they worked independently, any processes for obtaining or confirming data from study investigators, and if applicable, details of automation tools used in the process |
| Data items | 10a | List and define all outcomes for data which were sought. Specify whether all results that were compatible with each outcome domain in each study were sought (e.g. for all measures, time points, analyses), and if not, the methods used to decide which results to collect |
| 10b | List and define all other variables for which data were sought (e.g. participant and intervention characteristics, funding sources). Describe any assumptions made about any missing or unclear information | |
We hope you will find some of these guidelines useful. Remember, if you teach or supervise students alongside you research role, appropriate reporting informed by guidelines is standard practice in many fields – and we will only reach this position in energy if they are included in our teaching.
Banner photo credit: Kat Stokes on Unsplash
