Moving Beyond Factsheets: Framework Comparison/Synergies/Consolidation



In this Do-a –thon we want to discuss and (further) develop a concept for open-source (OS FW) framework comparison.

Here is why we would love to do this:

  • We want to consolidate the OS FWs and gain trust in their results and usability!
  • We want to know which OS FW is the most applicable for which specific task
  • We want to investigate in the argument being constantly stated within the community that
  • Parallel code development can be reduced through the open source concept
  • Finally, we want to identify synergies and propose concepts on how to use them

This is how the workshop agenda could look like:

  1. Ask ourselves ”What are the requirements for a comparative scenario following the rules of Open Science?”

  2. Introduce factsheets and discuss/add criteria to be compared (follow-up on this thread Breakout Group: Transparency - Model comparison - Factsheets)

  3. “What else needs to be done to allow comparison (connectors, common ontology (follow –up on this thread” from this we want to establish a regular Do-a-thon regarding comparisons and synergy identification

The results of this workshop can make an important contribution to open_MODEX- an ongoing three-year open science research project in which 5 OS FWs for energy system analysis are compared and synergies are derived.

We’ll be happy to receive your feedback and ideas!

Open modeling/environmental NGO bridge
Do-a-thon: running and testing open-source models

@sar_b Are you going to compare the communities as well as the frameworks? That would be interesting too. I don’t know which projects are on your list, but most seem to have evolved different norms, audiences, and levels of formalism.


@robbie.morrison I agree and this is definitely foreseen in the project and yet could be part of the criteria discussion at OpenMod Arhus. Regarding open_MODEX the 5 FWs being compared are urbs, oemof, GENESYS-2, OSeMOSYS, Balmorel. They are represented through their respective hosting institution.

Do-a-thon on COST grant for openmod activities

Koebler et al (2019) offer a very readable account of free software (often FOSS) development today and how it can be best supported. Smith (2018) writes specifically about scientific software in this context. The remaining references (below) are cited in the Koebler article and are classics in their own right. They provide useful background.


Eghbal, Nadia. (14 July 2016). Roads and bridges: the unseen labor behind our digital infrastructure. New York, USA: Ford Foundation. PDF version.

Koebler, Jason, Daniel Oberhaus, and Emanuel Maiberg (14 February 2019). The complicated economy of open source software. Motherboard.

Lerner, Josh and Jean Triole (March 2000). The simple economics of open source. USA: National Bureau of Economic Research.

Raymond, Eric S (2001). The cathedral and the bazaar : musings on Linux and open source by an accidental revolutionary. Sebastopol, California, USA: O’Reilly Media. ISBN 978-0-596-00108-7. Extended version of his essay originally published in 1998.

Smith, Nathaniel J (25 May 2018). The unreasonable effectiveness of investment in open-source infrastructure. njs blog.


Regarding points 2 and 3: I would argue that the definition of factsheets as consolidated ‘static sheets’ might be hard as there are a lot of heterogeneous communities with different (orthogonal) perspectives on the energy system leading to huge, confusing forms. (This is basically one of our lessons-learned from an attempt to define framework factsheets in open_eGo).
In SzenarienDB we are currently developing a collection of ontologies of energy scenarios and aim to define scenario factsheets as instantiations of parts of this ontology. You might be able to extend those ontologies for your purposes. There is also a proposal for an ontology session for the next workshop as a follow-up for a session on the previous workshop.
I am looking forward to your session :slight_smile: