Plot energy & carbon stacks

[danielolsen]

Purpose

Integrate energy & carbon bar stacks into codebase.

What is the code doing

plot_single_scenario() plots a single scenario. plot_two_scenarios() plots two scenarios.

Only fuels which are present in the grid (i.e. not 'geothermal' in Eastern) and generate a non-zero amount of energy (i.e. not 'wind_offshore' when we scale all generators to zero MW) are plotted. There are a few annotations with high-level statistics.

Time to review

Half an hour to an hour? I don't know how picky we need to be for plotting code. There are a few places where we could optimize the code for optimum DRY, but I think we can integrate these now and play around with refactoring later, if there are no glaring issues.

[BainanXia]

Maybe fuels = carbon_fuels + ['other' if 'other' in energy_ by_type' else []] + clean_fuels

[danielolsen]

Why single out 'other' here? The all_possible definition should get these in the right order, and line 31 should maintain order while dropping any that aren't present.

[BainanXia]

Previously, I was thinking removing 31-33 and fuels = carbon_fuels + clean_fuels, then I realize 'other' is not covered in this way. Nevermind, it's not critical.

[danielolsen]

Yes, unfortunately we don't have a good carbon intensity value for 'other' generators at the moment, so it's a nebulous category outside of 'clean' and 'carbon'.

[victoriahunt]

I'd suggest flagging analysis of 'other' generators carbon emissions as a Zenhub issue for future work. Right now it appears that all 'other' generators have 0 carbon emissions, which I think is misleading.

[danielolsen]

Done.

[BainanXia]

This type of bar plot contains key information that we would like to tell from our scenario results and they are beautifully generated. plot_single_scenario() and plot_two_scenarios() seem a little weird to me and should have better names. Also, I'm not sure we would like to limit the comparison up to two scenarios. It would be great to have a function that takes a list of scenarios from 1 to n and generates the bars side by side, however, this requires major refactoring of the current implementation and addressing the challenge of adaptive changing the parameters in the figure, such as bar size, locations, etc. I'm not sure whether we should proceed in its current form or keep this code in a notebook script until we can invest time to generalize it.

From this perspective, I would like bring @rouille and @kasparm into this conversation, given this is a common question for most of the plots currently in the spot check integration script.

[danielolsen]

@BainanXia I was having similar thoughts. I wrote the single-scenario version first, and when I started writing the two-scenario version I started thinking that it wouldn't be too much work to make this one take 1 to n scenarios instead.

Instead of having our dictionary of scenario-level results keyed by 'left' and 'right', we could instead have them keyed by a list of scenario IDs, and we maintain a list that keeps them ordered as they were input. It all depends on how we want to use these sorts of plots. @yixingxu might have some input too.

[BainanXia]

scenario.state.get_grid() and scenario.state.get_pg() are also called in generate_carbon_stats(scenario) as well and assigned to variables with same name. I could see two options to avoid duplicated calls, a) pass those variables into generate_carbon_stats, but this will affect the usage when the user calls generate_carbon_stats independently, b) check whether grid and pg are already assigned in the scope, if not load them as usual, otherwise just use it or make a copy. Maybe there are better ways to handle this situation and I'm more than happy to learn.

[danielolsen]

I saw this too. Perhaps this is a good use case for the ScenarioInfo object, and some more flexibility in the input arguments to some of our analysis functions. For each function that takes a Scenario object and will eventually load PG (big) and the Grid (a little slow to instantiate), perhaps we can make the input more flexible to also accept a ScenarioInfo object, and if it gets one, grabbing the PG/Grid from there instead of creating a new copy. Then the carbon bar plotting function could generate a ScenarioInfo object for each Scenario, and pass this object to e.g. generate_carbon_stats(). We would have to be careful about memory management though, since after this line we never need a full-dimension PG again, but it will remain in memory unless we del the ScenarioInfo object.

[rouille]

I think that if it is not too much work, it is better to generalize to n scenarios. And if you have the analysis/plot for n, then you have it for 2. I don’t see any drawbacks.

[danielolsen]

The latest refactor generalizes to 1-to-n Scenarios.

>>> from postreise.plot.plot_energy_carbon_stack import plot_n_scenarios
>>> from powersimdata.scenario.scenario import Scenario
>>> plot_n_scenarios(Scenario('403'))

>>> plot_n_scenarios(Scenario('403'), Scenario('469'), Scenario('511'))

[rouille]

Looks great.

[victoriahunt]

If you have very different scenarios being compared, and you base the carbon multiplier on the first scenario only, isn't it possible that the scaling could be suboptimal for the other scenarios? Wondering if carbon multiplier should be based on the average of the scenarios being considered.

[victoriahunt]

Confirmed that the bar plots look different depending on which scenario is selected first, due to the way carbon multiplier uses the first scenario listed only. When ratios of energy to emissions are very different in other scenarios being compared, this can create a situation where the first scenario selected has a large effect on the image that is generated:

[victoriahunt]

I'd like to request a demo notebook (in the demos section) to showcase how to use this function.

These are very nice looking plots and they will likely be a good type of bar plot (translated to Nivio) to show on the 'multiple scenario comparison' part of the website's dashboard. The functionality to show as may scenarios as you'd like side by side is very useful.

[danielolsen]

@victoriahunt I can do that. Do we have a way of tracking which files/functions have associated demo notebooks? Down the road, if someone changes something in this script, but doesn't know to change the demo notebook, the two will drift apart. Maybe a comment or docstring at the top of files which have associated demos?

[victoriahunt]

Do we have a way of tracking which files/functions have associated demo notebooks? Down the road, if someone changes something in this script, but doesn't know to change the demo notebook, the two will drift apart. Maybe a comment or docstring at the top of files which have associated demos?

We don't currently track this but you're right that we should. When there was the refactor /reorganization of PostREISE a while back where scripts got reorganized into different folders I realized a bit later that my demo notebooks didn't work because they imported functions using the old folder system, so I had to edit accordingly. Comment at top of file about associated demos seems smart.

[danielolsen]

A demo notebook has been added, and the module points to it in a top-level comment.

[rouille]

You can use an hyperlink here:
`demo notebook https://github.com/intvenlab/PostREISE/tree/develo/postreise/plot/demo/plot_carbon_energy_carbon_stack.ipynb`__

It anticipates the fact that the branch will be merge into develop

[danielolsen]

To me, directing to the folder path seems cleaner than directing to github. Plus, that would be a loooooong line.

[BainanXia]

All the comments are addressed. Thanks for being responsive!