Social impacts of peer-to-peer energy trading: a rapid realist review protocol

Michael Fell

This protocol outlines our approach to conducting a rapid realist review to identify evidence for potential impacts on people and society of peer-to-peer energy trading (and of distributed ledger technology used in this context). Our motivation for the study is to help anticipate who might stand to win or lose (and how and why), inform policy/regulation to help maximize benefits and minimize harm, and identify research gaps.

While our focus is in the energy sector, we also plan to draw on evidence (where relevant) from examples of sharing economy models in non-energy sectors. We have already developed and engaged around a provisional programme theory (presented as a set of Context-Mechanism-Outcome statements), which we will develop as the review progresses. We set out where and how we will seek to identify evidence (through online searching, reference checking and calling for evidence). In line with our exploratory and iterative approach, we propose broad inclusion criteria. We will assess evidence quality subjectively on the basis of relevance and rigour for each Context-Mechanism-Outcome group, not at document level. Synthesis will be achieved through developing our programme theory and connecting evidence to it. We will disseminate findings through an academic paper (or papers), one or more policy briefings (with associated engagement events), one or more public blogs, and materials will be openly shared on an ongoing basis through an Open Science Framework page.

Introduction

Peer-to-peer energy trading (where prosumers trade energy directly with each other) is the subject of much industry, policy and research interest because of the potential it holds to optimize use of distributed energy resources (DER) and energy networks. It is also currently the most common use case for startups in the energy-blockchain space (see Lacey [2018]). If widely adopted it would represent a significant move away from the status quo in established energy markets. For this reason it is necessary to anticipate and understand the positive and negative effects it could have on people and society This document is a protocol for a rapid realist review which aims to respond to this need. Realist reviews are appropriate for examining complex interventions and can help provide “an explanatory analysis aimed at discerning what works for whom, in what circumstances, in what respects and how” (Pawson et al., 2005: 21). Rapid realist reviews aim to do this in a way which is capable of “responding to time-sensitive and/or emerging issues, while preserving the core elements of realist methodology” (Saul et al., 2013: 1). The protocol specifies the steps which will be taken in the review. Since realist review is inherently an iterative, learning approach, this protocol will provide a starting point which is expected to evolve as the review progresses. This protocol is being written after one stage – the production of provisional Context-Mechanism-Outcome statements (see below for summary) – has been completed, and therefore focuses on the evidence identification and synthesis stages. Please note that we1 have kept this protocol relatively brief in order to get the evidence identification stage underway as quickly as possible. Because we are tending to take an inclusive approach, and because of the ability to iterate and adapt during the course of the review, we think this will best balance the requirement to get early feedback and produce rapid learnings with methodological best practice. We intend to produce future documents which go into more detail on the background, motivations, justification for scope, etc. and will share these when available (see dissemination section for more details of outputs).

Scope

For the purposes of this review we are interested in impacts as they might be experienced by people as individuals and as part of families, households, communities and societies. This includes economic impacts such as energy bills or revenue from selling energy, as well as factors such as way or life, culture, community, health/wellbeing, and environment (as it directly impacts people). [Based on the International Association for Impact Assessment page on Social Impact Assessment].

Regarding the concept of peer-to-peer energy trading, we initially interpret this broadly to include ‘true’ peer-to-peer arrangements, where there is some direct energy-related transaction between individuals, as well as an array of community-based, local and peer-to-market arrangements that fall short of simply selling energy to a large-scale (e.g. national) network without any form of matching [This encompassing approach will be considered again depending on the quantity of relevant evidence that is identified]. Given that the expect the evidence base in this area to be limited, we will err towards taking an inclusive approach. We expect the vast majority of identified evidence to relate to electricity trading, but will not exclude other energy vectors or services (e.g. gas or heating) if examples are identified.

Recognising the limited evidence base, we also propose to expand the scope of the review to include evidence on social impacts of sharing economy models more broadly, and across sectors. We do not expect this exercise to be comprehensive, but through the approach outlined below we will seek to identify the most relevant examples to the energy sector. Figure 1 summarises the scope of the review.

While by no means all peer-to-peer energy (or other) trading arrangements are enabled by distributed ledger technologies (DLTs), we will pay particular attention to any impacts which appear to be directly associated with the involvement of such technology. This will inform consideration of the extent to which DLTs are should be viewed as a positive part of future services.

In a final point on scope, we intend the outputs of the review to be most relevant to countries (such as the UK) which already have substantial established national energy infrastructure. We anticipate the issues related to operating peer-to-peer models in these contexts will be different to those connected with more basic energy access/reliability in countries without established infrastructure. However, this does not mean that we will exclude evidence drawn from examples in countries without substantial established infrastructure – rather that we will consider whether it is sufficiently relevant to provide learnings for this context.

Aims and research questions

The overall aim of this project phase is to identify the range of, and weight of evidence for, possible impacts on people and society associated with DLT-enabled peer-to-peer energy trading. The research questions are:

1. What may be (a) the outcomes of introducing P2P energy trading, (b) the mechanisms by which these outcomes may come about, (c) the contexts in which these mechanisms may or may not operate, (d) the relative importance of outcomes and (e) the likelihood of outcomes?
2. What is the evidence that expected social impacts are occurring (or could occur), how, and in which contexts?
3. How, if at all, are social impacts being monitored and measured?
4. Which social impacts are (a) expected and (b) demonstrated to be associated with the use of distributed ledger technologies, and what are the mechanisms/contexts for this?
5. What actual or proposed policy/practice responses exist to maximise social benefits and minimize harms?
6. Where are the key gaps in the evidence regarding social impacts that future research should address?

The purpose of this research is to inform mitigation of negative impacts (especially for otherwise vulnerable groups), maximization of benefits, as well as to recognise and prioritise areas for regulation and future research in this space. Aside from its intrinsic importance, considering social impacts is also a requirement of the Social Value Act 2013 so will be a consideration for UKs public bodies offering and procuring peer-to-peer trading. To make the research as relevant as possible in the short-term we aim to identify specific contexts where the introduction of peer-to-peer trading is likely to come with maximum benefit and least harm, so that these can be targeted first.

Development of programme theory

The first stage of a realist review involves production of hypothesised contexts, mechanisms and outcomes (CMOs) which together make up the programme theory [we use this term provisionally and further discussion of the precise meaning of this as compared to Theory of Change and logic model approaches will be reserved for future documents].

These CMOs may be represented diagrammatically or spelled out in the form of CMO statements. We developed initial CMOs for this project on the basis of informal reading and discussion with individuals representing government, the energy regulator, consumer and community energy organisations. These provisional CMOs are available online here, and we also include CMO statements in the appendix. The CMOs will be updated on a regular basis throughout the review the reflect the evidence we have identified, as these updated versions will be made available at the link above.

Identifying evidence

It is not possible to specify fully in advance the full process by which a realist review will be conducted. Such reviews are iterative by design (Pawson et al., 2005), and should adapt to follow the evidence as the review progresses. This is (perhaps especially) the case in a rapidly developing area such as this, where an exploratory approach is most appropriate. However, we hope that provision of an initial plan will be useful here in giving readers an idea of the coverage we expect to achieve, and in soliciting feedback on the approach. This and all subsequent sections are therefore subject to variation depending on emerging findings of the review.

We intend to seek evidence in a number of ways: through database and other online searches; through following references; and through direct engagement with individuals and organizations working in areas relevant to the review. To inform online searches, we have identified a number of key conceptual areas within this review: peer-to-peer and sharing economy; social impacts (including concepts around people and concepts around impacts); and energy. Table 1 sets out the sub-concepts we would like to include, and how we intend to operationalise these in searches.

Table 1: Concepts, search terms and example search string.
	Peer-to-peer	People	Impacts	Energy
Concept	Peer-to-peer trading Community self- consumption Transactive [energy] Sharing economy Collaborative consumption Collaborative economy Peer economy Platform economy	People Society Social Consumers Prosumers Customers	Distributional Equity Equality / inequality Fairness Justice Poverty Vulnerability Protection Lifestyle Culture Health Quality of life Bills Socio-economic	Energy Electricity Power (NB – this term is also commonly used outside of the energy context so use will be kept under review)
Search term	“peer-to-peer” “peer to peer” p2p “commun* self- consumption” transactive “sharing economy” “collaborative consumption” “collaborative economy” “peer economy” “platform economy”	people societ* social* *sumer* customer*	distributional *equit* *equal* fair* justice *pover* vulnerab* protect* lifestyle health* “quality of life” bill* socio*economic	energy electricity power
Scopus example	( TITLE-ABS-KEY ( “peer-to-peer” OR “peer to peer” OR p2p OR “commun* self- consumption” OR transactive OR “sharing economy” OR “collaborative consumption” OR “collaborative economy” OR “peer economy” OR “platform economy” ) ) AND ( TITLE-ABS-KEY ( people OR societ* OR *sumer*  OR customer* OR distributional OR equity OR fair* OR justice OR protect* OR lifestyle OR health* OR “quality of life” OR bill* OR socio*economic) ) AND ( TITLE-ABS-KEY ( energy OR electricity) )

As depicted in figure 1, the search will have a primary and a secondary focus which can be expressed (with reference to the concepts in table 1) as follows:

1. Peer-to-peer AND Energy AND (People OR Impacts)
2. Peer-to-peer AND Impacts

Since pilot searches on academic databases reveal a manageable number of publications focusing specifically on P2P energy trading, searches in stream 1 (containing the ‘energy’ concepts) will include the widest variety of sharing economy, people and social impacts terms where possible. Searches in stream 2 (not constrained to energy) will not contain generic social terms (e.g. people, society) but instead focus on more specifically impact-related terms. This is because pilot searches suggest that otherwise we would identify an unmanageably large number of potential sources.

The following bibliographic databases will be searched:

• Scopus
• Web of Science (all databases)

Searches will also be developed based on the above search terms for following: [Where Google is used in searching, results pages will be read with the inclusion criteria in mind, and those results which the title/brief description indicate will likely meet the criteria will be saved. Where there are a large number of potentially relevant search results, we may additionally specify filetype:PDF as previous experience suggests that project reports are often made available in this file type].

• Government publications (using Google, restricting search to site:.gov.uk, .gov, .gov.au, etc.)
• Non-profits, civil society organizations, etc. (using Google, restricting search to site:.org, .org.uk, .org.au, etc.)
• Academic institutions (using Google, restricting search to site:.ac.uk, .edu, etc.)
• Google Scholar, using Publish or Perish software and downloading the first 1000 results (Bramer et al., 2016; Haddaway et al., 2015).
• Cambridge Energy Policy Research Group working papers
• UK Energy Research Centre
• European Commission Research and Innovation (Energy)
• US Department of Energy (including SciTech Connect)

A general Google search with more focused terms from the above table.

In all cases the specific search string, date and number of results will be recorded. All hits from database searches will be downloaded and imported in reference manager software (Zotero). For searches of other sources, results will be downloaded where the title and/or initial screening suggests the document is likely to meet the inclusion criteria.

In addition to the above searches, the reference lists of sources passing the inclusion criteria below will be read, and any documents considered likely to be relevant on the basis of the bibliographic information will be added to the database and screened. Depending on the number of sources included, forward citation checking will also be employed (using Google Scholar, downloading references which are thought likely to meet the screening criteria.

We will also proactively engage with others working in this area in case they are able to share any unpublished relevant evidence. We will achieve this by emailing individuals whom we are aware are working in this area, and by putting calls to share evidence on social media such as Twitter.

Inclusion screening

All references will be imported into the systematic review management software EPPI-Reviewer 4 and de-duplicated. They will then be screened by a single reviewer on the basis of the inclusion criteria below. A first-pass screening will be done on title/abstract, with a second screening on the full document. The inclusion criteria are as follows, all of which must be met (except where noted otherwise):

• Substantial consideration of sharing economy or related models (this to be interpreted broadly when in doubt)
• Contains empirical or model-based evidence, or reasoned conceptual/theoretical consideration, of impacts on people or society
• In context of energy (search 1 only)

The reference lists of all documents still included at this point will be checked, and any relevant documents referred to will also be loaded into EPPI-Reviewer and passed through the screening process.

Quality assessment

It is expected that the types and sources of evidence drawn upon in this review will be highly heterogeneous. Rather than employing multiple quality appraisal checklists, this review will follow the approach recommended by (Pawson et al., 2005) and rely more heavily on subjective judgement as to the relevance of the evidence to the subject of the review, and its rigour. Furthermore, this judgement will not be made for studies in their entirety but for each evidential claim as it relates to a specific part of the developing programme theory. It is anticipated that low/medium/high ratings will be assigned for relevance and rigour. Where external experts recommend the use of specific documents, these recommendations will be taken into account when considered relevance/rigour (as suggested by (Saul et al., 2013)).

Extraction

Basic information such as location, date and sector (see below) will be extracted from each, which will allow generation of a simple quantitative overview of the included sources. The main extraction will be in the form of open-ended text summaries describing key outcomes, mechanisms, context and recommendations (see next section). We will set up the following codes in EPPI-Reviewer but expect this to be adapted as appropriate as material is reviewed.

• Geographical location of study (country list)
• Date of deployment (>=2015, 2010-2014, 2005-2009, 2000-2004, <=1999)
• About the source (tick all that apply: does it include: qualitative, quantitative, modelling/simulation, primary data collection/analysis, secondary data analysis, review/meta-analysis, primarily theoretical/conceptual, comments [description of study including points such as sample size and representativeness, specific methods, if appropriate])
• Sector (energy, accommodation, transport, other [more to be added if encountered regularly])
• Characteristics of peer-to-peer operation(s) (if applicable)
• Types of impact considered (economic, lifestyle, cultural, community, quality of life, health, other)
• CMO summaries (open text summaries describing the main outcomes discussed, the mechanisms by which they are suggested to arise, contextual factors, practice recommendations, each with low/medium/high rating for relevance and rigour).
• Is distributed ledger technology employed (yes/no – if yes, include details and CMOs connected directly with DLTs).
• Quantitative associations. Where sources give some indication of a level of association between two or more relevant variables, this will be noted to inform future graphical probabilistic modelling.

Analysis/synthesis

We will use EPPI-Reviewer to conduct basic quantitative analysis to describe the breakdown of the included sources. For the main analysis, source bibliographic information and extracted CMO summaries will be imported into NVivo and subject to qualitative content analysis (Hsieh and Shannon, 2005). The approach will primarily be inductive (i.e. subjects/themes will be sought and coded for in the text), but, where appropriate, codes based on the provisional CMOs will also be applied.

The codes will be used to identify evidence which supports or undermines the provisional CMOs, and which suggests new ones. The synthesis stage will involve mapping evidence onto the provisional CMO table, and then iteratively producing new tables featuring amended and additional CMOs that reflect the available evidence. The final product will be an augmented CMO table that includes the CMOs, along with relevant summaries of supporting evidence (which references and links to the original source) and flagging of gaps. A colour coding system to indicate where there is strong, moderate or weak/no evidence will be employed if appropriate.

Dissemination

We intend to write an academic paper with a fuller explanation of the background, motivations and process of the review, as we think there will be value to the community in setting out more detail on the approach as we are applying it. We will also produce an academic paper on the findings. We anticipate presenting the results at academic conferences.

Looking beyond academia, provisional summaries will be shared and discussed with the original advisors based in government, the regulator and consumer/community organizations. Following this engagement we will produce a policy briefing and explore opportunities to run an engagement event (such as a workshop) around this. We will produce blog posts summarizing the findings and share these as widely as possible. We will seek ways to tie the findings in with other engagement projects we have underway (such as the development of a peer-to-peer energy trading board game).

The project has an Open Science Framework site where project documents (including CMO statements) are, and will continue to be, made available. We will explore ways or visualising the programme theory in a way that clearly highlights the weight of evidence supporting it (ideally with direct links to that evidence).

If you would like to discuss dissemination opportunities, or any other aspect of the project, please contact Michael Fell (michael.fell@ucl.ac.uk).

Banner photo credit: Alireza Attari on Unsplash