Indicators and common factors that help to determine the impact of EU projects promoting energy efficiency

A study into 41 EU-funded energy efficiency projects across Europe looked at primary energy savings, greenhouse gas (GHG) savings, investment triggered, market stakeholders with increased skills on energy issues, and renewable energy generated.

The projects received €58.5m of funding from the EU’s Intelligent Energy Europe II (IEE-II) and Horizon 2020 (H2020) programmes to focus on increasing energy efficiency in EU companies, and have led to €500m of investment within the organisations that took part and saving 3.5TWh per year of primary energy, as well as a 1.1 MtCO2/year GHG reduction.

The review was carried out by Ricardo, a global engineering, environmental and strategic consultancy, for CINEA, the European Climate, Infrastructure and Environment Executive Agency.

Energy efficiency in industry has a key role to play in the EU meeting its 2030 and 2050 climate targets and fulfilling its objectives under the Paris Agreement. Each of the 41 projects was evaluated individually, and the impacts (reliable and acceptable only) totalled across the portfolio. The results show the impact of making changes within organisations and what a difference they can make to us all achieving our common goal of reduced emissions.

One of the main tools to measure a project’s success, and to identify how project activities resulted in real impacts in terms of reducing company energy consumption and GHG emissions, are the Key Performance Indicators (KPIs). The review team encountered inconsistency across the portfolio in how projects reported their KPIs, if at all. This made it challenging to identify what went well and what could be improved. To address this, the study team carried out a thorough analysis of the projects’ calculations and restructured them using input from stakeholder interviews, common indicators and literature sources.

As a first step, the activities, outputs and achievements of each of the 41 projects were mapped to ensure a comprehensive understanding of each project. Next, the projects’ calculation steps and assumptions that led to the KPIs were assessed to identify data gaps and steps that could be enhanced. Every step and overall KPI was assigned a rating of ‘reliable’, ‘acceptable’ or ‘uncertain’ based on this assessment.

To allow a homogenous re-estimation of the KPIs across all projects, a standardised approach and format was used. This involved disaggregating information and calculation steps made by the projects, where necessary. Through this approach, a catalogue of common steps, methods and assumptions used was developed. This allowed identification of common factors that had been applied to the KPIs reported by the projects. 

The re-estimation methodology was developed so that it was flexible enough to fit most project’s calculations and so that it was comparable across projects. This approach was driven primarily by the wide variety of approaches taken by the projects in their calculations and also by the different types and amounts of information available on project impacts. 

In the cases in which the KPIs were already deemed to be reliable, the calculation was restructured to fit the standardised format. Where the KPIs were not reported or reported with missing assumptions and steps, the standardised approach allowed the study team to estimate the impacts or KPIs generated by an activity for each project. For example, how much energy was saved as a result of an energy audit. Using the same harmonised calculation methodology across the projects allowed for better comparison of the impacts.

Literature values and interviews were used to fill data gaps, improve the understanding of project impact calculations, identify further impacts of projects and establish the extent to which the impacts of completed projects had been sustained and replicated. The interviews, carried out mainly with project coordinators and partners, facilitated the understanding of the activities undertaken and challenges faced by each project, as well as any lasting impacts that extended beyond the project lifetime.

The re-estimation process allowed the study team to identify a number of common factors that can be carried forward into future analyses and estimations of impacts. The most significant finding was the energy savings rate achieved through different activities. An average audit carried out by the projects yielded 4.5% in energy savings for participating companies, while capacity building activities and tools or benchmarks yielded 4.1% and 3.2%, respectively. Moreover, further analysis of more than 3,500 audits carried out showed that an average audit identified 18% in potential energy savings. On average, 25% of those potential energy saving measures were implemented by the companies.

Furthermore, the approach taken allowed the study team to disaggregate the collected data and KPIs by type of action or activity leading to the respective impact. Noticeable differences identified were that the energy savings from IEE-II projects relied more on capacity building-related activities, while H2020 projects relied more on audit-related activities. Comparing the impacts of activities during and after the project lifetime, it was noted that audits led to a higher percentage of the energy savings during the project time – for many projects, a significant number of audits was undertaken during the project lifetime and relatively few after – while capacity-building activities and tools/benchmarking had effects after the project lifetime. Tools and benchmarking are more relevant for projects that focus on a single sector compared to projects taking a cross-sectoral approach.

The challenges encountered in carrying out and subsequently assessing these projects included the engagement of SMEs, data collection and quality, gaps identified in project reporting, particularly for older projects where interviews were less likely to be possible to fill such gaps, and information gaps with regard to the non-economic benefits of energy efficiency improvements. The study therefore sought to draw together recommendations to not only address these challenges in future programmes, but also to enable higher-quality evaluations of these programmes. Providing a standardised template or format for presenting the impact chain from activity or action to impact could provide greater certainty with regards to how the impacts were estimated and what sets a project apart from another.

It is clear that ongoing support through funded programmes remains necessary to ensure progress continues in the area of energy efficiency for industry and services in the EU, as does highlighting the benefits achieved by individual projects through communication campaigns. Fostering synergies with other EU funds and schemes to track the actual implementation of the recommended energy saving measures at a project level would yield significantly more concrete evidence for the impacts of these projects, and the evidence necessary to share the successes and influence other European SMEs to take energy efficiency seriously and make changes. Coordination and support activity projects focusing on promoting energy efficiency improvements within SMEs evidently have significant contributions to make towards the 2030 and 2050 targets, yet a large gap remains to achieve such targets. 

This assessment was carried out by Ricardo PLC, a global engineering, environmental and strategic consultancy for CINEA, the European Climate, Infrastructure and Environment Executive Agency. For the full report please see here.