Personal report from the openmod Munich workshop panel discussion
A panel discussion was held on Friday 13 October 2017 with two representatives from ENTSO‑E as part of the 7th openmod workshop. Some personal impressions of that discussion and the subsequent breakout group follow. Tom Brown arranged the session, so kudos to Tom.
ENTSO‑E stands for European Network of Transmission System Operators for Electricity and is the umbrella organization for national and sub-national transmission system operators (TSO) across Europe. ENTSO‑E coordinates the Transparency Platform (TP), operated under European Commission Regulation 543/2013 (a full reference is given elsewhere on this thread). Although ENTSO‑E is cited throughout this post without qualification, all remarks derive from the two representatives present and may not necessarily reflect official ENTSO‑E positions.
ENTSO‑E is guided by its member TSOs and its policy positions normally require unanimous agreement. Most ENTSO‑E data is sourced from the TSOs who, in turn, may collect it from various power exchanges (PX) and market participants within their jurisdictions. Hence, ENTSO‑E sometimes faces an uphill battle to obtain good primary data and encounters many of the same issues that our community does when trying to create consistent and complete datasets. Open licensing can only take place with the consent of the primary data owner. In other words, the licensing web starts with the copyright holders.
Thanks to concerted efforts within our community, including by the OPSD project (Lion Hirth), the OpenEnergy Platform (Ludwig Hülk), and the PyPSA project (Tom Brown), ENTSO‑E are now aware that, in relation to the Transparency Platform and ENTSO‑E public data more generally:
- the open licensing of public datasets is essential for both scientific research and open development
- data quality remains an issue and some means by which missing and spurious data points can be flagged and fixed at the point of publication would be to everyones’ advantage
ENTSO‑E noted that the Transparency Platform was conceived for transparency and not as a source of modeling data. However the Transparency Platform does get regularly interrogated and scraped, leading to degraded service. Anecdotally, some TSOs source modeling data from the platform rather than using internal routes.
ENTSO‑E is working on its Pan-European Market Model Database 3.0 (PEMMD3), an extension of version 2.0 used internally, which is distinct from the Transparency Platform. The extent to which this database will be made public and under what licensing terms has not been determined.
The European Commission Integrated Database of the European Energy Sector (IDEES) project was mentioned. But similarly the extent and terms governing its public usage have yet to be notified.
In contrast, the BNetzA (Bundesnetzagentur or German Federal Network Agency) SMARD (Strommarktdaten und zielt auf mehr Transparenz im Strommarkt) energy market platform releases its information under a Creative Commons CCÂ BYÂ 4.0 permissive license (congratulations to BNetzA).
During the breakout group, Frauke Wiese described the OPSD project, Ludwig Hülk the OpenEnergy Platform, and Jonas Hörsch work he and Fabian Hoffmann had undertaken on power plant fleet comparisons. A python module named powerplantmatching can be used to combine disparate power plant fleet databases (vertical matching) and then compare and flag discrepancies (horizontal matching). The module will also optionally select the median value when faced with conflicting plant attributes (field values). I think it would be fair to say that the ENTSO‑E representatives were not much aware of the scope and quality of the work on database support and data sourcing and cleansing taking place within the openmod community.
Information categorization was a recurring theme. For example, some power plant fleet databases distinguish between lignite and hard coal firing while others do not. Simple aggregation is not a difficult issue, but misaligned definitions are. This means that a broadly agreed energy system glossary would be advisable. The openmod community has made a start on their wiki, but this work remains highly incomplete. Database ontologies were also discussed. Wiktionary defines an ontology as “a structure of concepts or entities within a domain, organized by relationships; a system model”. A glossary can be seen as a precursor to a coherent ontology.
Obtaining grid information presents problems for energy modelers, particular geo-location data for engineering assets such as substations. TSOs will not allow too detailed information at the substation level to be made public for security reasons, so there was some discussion on adding white noise (say ±10km) to latlon information to address this concern. It was pointed out that such information will doubtless be available on OpenStreetMap or related websites in due course.
Finally, data and metadata standards were not much discussed but remain an important issue on which to seek consensus and adoption. The Common Grid Model Exchange Specification (CGMES) technical standard for network modeling was mentioned.
My personal thanks to the two ENTSO‑E representatives for contributing to the discussions on open energy system data and to ENTSO‑E more generally for entering into a dialog with the openmod community regarding such data.
â–˘