Formatting of the oemof-solph results

[Bachibouzouk]

Context

Let's consider an oemof-solph model. The specific components within the model, nor the solver do matter at this stage as we would like to inquiry about the result processing in general.

A typical workflow to build and solve an oemof model would be the following:

from oemof.solph import EnergySystem, Model

es = EnergySystem(...)
# add components and constraints to the energy system
m = Model(es)
m.solve(...)  # provide solver information here

results = m.results()

Currently results is a dict with

1. flows (bus object, component object ) or (component object , bus object) tuples as keys
2. nodes (mainly only storage component) (component object, None) tuples as keys

We are looking at the flow results (i.e 1.), in this case the values within the results dict are are dictionaries and have two items with scalars and sequences as keys.

scalar contains typically an invest if the component was subject to an Investment (optimization). It is not easy form the oemof-solph docs to know what other values could be expected here. It seems that it contains an empty pandas.Series if not subject to an Investment.

Problem

Oftentimes what is wished to perfom techno-economic analysis of energy systems requires to perform calculations using the flows, the invested capacities as well as some parameters which were used as inputs (such as existing capacities, annuity costs, variable_costs, etc. ). There is currently no function that fetches the most commonly used costs parameters.

Solutions

The aim of this github discussion is to look at possible solutions for OPTIMG project but also to see if some solution could be already be partly implemented at the oemof-solph level as many oemof users are performing those calculations. This could help address the current "lack of love" given to result processing as expressed by one of the oemof-solph core devs.

[Bachibouzouk]

Essentially all information we need could be formatted in a tidy way, with a multi index

                                                                                                                                  2024-01-01 00:00:00  2024-01-01 01:00:00  2024-01-01 02:00:00  2024-01-01 03:00:00  investments
bus                            energy_vector direction asset                                  asset_type             oemof_type                                                                                                  
ac_bus                         Electricity   in        diesel_generator                       diesel_generator       transformer             8.666370            11.402538            12.188092                  NaN    12.188092
                                                       inverter                               transformer_station_in transformer            -0.000000            -0.000000            -0.000000                  NaN    -0.000000
                                             out       ac_bus_excess                          excess                 sink                    0.000000             0.000000             0.000000                  NaN     0.000000
                                                       electricity_demand_ent_critical (@)    reducable_demand       sink                    0.234423             0.528340             0.660841                  NaN          NaN
                                                       electricity_demand_ent_noncritical (@) reducable_demand       sink                    0.000000             0.000000             0.000000                  NaN          NaN
                                                       electricity_demand_hh_critical (@)     reducable_demand       sink                    8.120279            10.623780            11.293251                  NaN          NaN
                                                       electricity_demand_hh_noncritical (@)  reducable_demand       sink                    0.000000             0.000000             0.000000                  NaN          NaN
                                                       electricity_demand_pf_critical (@)     reducable_demand       sink                    0.311668             0.250417             0.234000                  NaN          NaN
                                                       electricity_demand_pf_noncritical (@)  reducable_demand       sink                    0.000000             0.000000             0.000000                  NaN          NaN
dc_bus                         Electricity   in        battery                                bess                   storage                 0.000000             0.000000             0.000000                  NaN     0.000000
                                                       pv_plant                               pv_plant               source                  0.000000             0.000000             0.000000                  NaN    -0.000000
                                             out       battery                                bess                   storage                 0.000000             0.000000             0.000000                  NaN     0.000000
                                                       dc_bus_excess                          excess                 sink                    0.000000             0.000000             0.000000                  NaN     0.000000
                                                       inverter                               transformer_station_in transformer             0.000000             0.000000             0.000000                  NaN          NaN
diesel_bus                     Gas           in        diesel_fuel_consumption_period (@)     gas_dso                transformer            34.665479            45.610150            48.752370                  NaN    48.752370
                                             out       diesel_bus_excess                      excess                 sink                    0.000000             0.000000             0.000000                  NaN     0.000000
                                                       diesel_fuel_feedin_period (@)          gas_dso                transformer             0.000000             0.000000             0.000000                  NaN          NaN
                                                       diesel_generator                       diesel_generator       transformer            34.665479            45.610150            48.752370                  NaN          NaN
diesel_bus_consumption_pdp (@) Gas           in        diesel_fuel_consumption                gas_dso                source                 34.665479            45.610150            48.752370                  NaN    48.752370
                                             out       diesel_fuel_consumption_period (@)     gas_dso                transformer            34.665479            45.610150            48.752370                  NaN          NaN
diesel_bus_feedin_pdp (@)      Gas           in        diesel_fuel_feedin_period (@)          gas_dso                transformer             0.000000             0.000000             0.000000                  NaN     0.000000
                                             out       diesel_fuel_feedin                     gas_dso                sink                    0.000000             0.000000             0.000000                  NaN     0.000000

here the levels of the multi-index are

1. bus name
2. energy vector or carrier (such as "electricity", "gas" etc.)
3. direction of the flow relative to the bus ("in" or "out")
4. name of the component (here named asset instead of component)
5. type of the asset, this would be analogous to name of the facade
6. oemof_type, this should simply be what comes out of type(< an oemof component>)

But this list can be expanded or reduced based on discussions.

The reason for laying the timeseries in this fashion (as opposed to a transposed version) is because then each column of the dataframe is of the same dtype

With this data structure we could imaging adding columns such as "existing capacity", "annuity", "efficiency", "variable_costs" and have all the information necessary to perform all of the common costs calculation.

As the number of "extra" columns ("investments", existing capacity", "annuity") is small it is not a burden to keep a list of those column names to easily get only the flows out of this dataframe by excluding the columns named after this list.

[Bachibouzouk]

This is potentially already formated this way in the Calulator class in oemof.tabular

[Bachibouzouk]

oemof_results_structure.ods
oemof_results_structure.xlsx

The headers can be sorted into some categories:

• index (minimal information to make each row identifier unique, some of this information could also go into RAM inputs category)
• RAW ouputs (what oemof.solph returns)
• postprocessed RAW outputs (minimal operations on the sequence, like min, max, sum, average etc...)
• RAW inputs (list of inputs needed for the calculated outputs, like marginal cost, renewable factor of an asset, etc)
• calculated outputs

The calculated output categorie could be structured in the following way

|column name |python function name which computes row level operations| description of the output parameter | list of required columns for this output to be calculated |

This way one could document the outputs as they add them. Hopefully most of the fonctions acting row-wise should be easy to understand to a person looking at the code later. This structure allows also someone to add to the mapping a new function where the costs are calculated in a different way.

The column names can be sorted to their categories, making it easy to display only part of the dataframe if needed

[henhuy]

This is potentially already formated this way in the Calulator class in oemof.tabular

I tried to implement such functionality using the Calculator class and already used it within digiplan.
See results implementation L802 and following here:
https://github.com/rl-institut/digiplan/blob/9868e962d790a4b7318bddc2558469bae89d7ed5/digiplan/map/calculations.py#L802

My idea was to have a clean way of calculations which can depend on each other (like a chain of dependencies) and can be stored in database as well. The solution works, but I feel like I overimplemented it...
On the other hand, having multindex-dataframes are not so handy as well IMO and also have its limitations calculating all desired results.
But maybe we could come up with a mixed solution?

[Bachibouzouk]

If I am not mistaken the Calculator class has attributes which are Multi-indexes DataFrames, no?

I understand more the Calculator each I look again at it. I wouldn't say over-implemented, but under-documented :). The problem I often face is I don't know what is possible and how I should use it at best. I understood I need to use what is called "dependencies", but how should I refer to them, are there some builtin ones?

On the other hand, having MultIndex-DataFrames are not so handy as well IMO and also have its limitations calculating all desired results.

Do you have an example of a calculation which is not implementable on a Multi-Index Dataframe?

But maybe we could come up with a mixed solution?

I think it is a good idea :)

[Bachibouzouk]

To me an important requirement is that it should be easy for a lay person to understand how something is calculated and to be able to create a new calculation easily. Because as I learned, if you ask 3 economists how to calculate the cost of something, you often get 3 different answers

[ciaradunks]

Defining the desired parameters for the results

Below I have listed all of the results that I think should be available to the user. Once they have been properly implemented in the new results structure (using the tidy dataframe as origin), they should be ticked off. I have tried to order the results depending on how they are found and whether they can be accessed directly or need to be calculated.

List of results from model directly:

• Optimized capacity/investment (and optimized max input/output for storage)
• Variable costs in/variable costs out
• Flows in/flows out

Lists of desired parameters to be calculated from model results:

• Total system costs (from optimization only)
• Specific system costs (total system costs / total demand)
• Aggregated flows
• Storage level?
• LCOE per component (either with optimized capacitiy or existing + optimized capacity / output of component)
• Excess energy per bus
• Fuel consumption per source

List of desired parameters from inputs directly to be saved in results:

• Existing capacity (and existing storage capacity)
• Maximum capacity/capacity potential (and storage capacity potential)
• Max input/output (capacity potential) (only for storages)
• Minimum capacity (and storage min)
• Optimizable/expandable
• Annuity (CAPEX + fixed O&M) (capacity cost and storage capacity cost)
• CAPEX (if defined)
• Opex (fixed) (if defined)
• Lifetime (if defined)
• Renewable factor (if defined)
• Emission factor (if defined)
• Land requirement (if defined)
• Water footprint (if defined)

List of desired parameters to be calculated from inputs and model results:

• Total capacity (to be calculated from existing capacity input and optimized capacity)
• Upfront investments (to be calculated from capex input and optimized capacity)
• Renewable share of system (to be calculated from renewable generation / total generation)
• this requires the renewable factor input, which is set to 1 or 0 and multiplied by all generation flows
• only generation components should have the renewable factor set
• Total emissions (to be calculated from emission factor input and flows from generation components)
• Specific emissions (to be calculated from total emissions and total demand)
• Emissions per power source (calculated from emission factor * flow of each power source)
• Additional land requirement per component (calculated from land requirement * optimized capacity)
• Total land requirement per component (calculated from land requirement * total capacity)
• Water footprintper component (calculated from water footprint * flow)
• Total land requirement (to be calculated from land requirement input and total capacities)
• Total water footprint (to be calculated from water footprint input and water flows)?

List of desired parameters specific for multi-nodal energy systems:

• Demand per node
• Renewable share per node
• Emissions per node
• Excess elec. per node
• Electricity transmission per node