AnalyzePG stack plots are fragile due to 'negative curtailment'

[danielolsen]

There is often some amount of 'negative curtailment' in our results. I think this is due to our use of the barrier method for solutions (with no time-consuming crossover step at the end), so there is sometimes a small fractional amount more generation from renewable resources than the profile would suggest is possible. However, ANY negative value, no matter how small, makes the stack plot error out. I imagine we could fix this fairly easily with a np.clip call.

Example:

>>> from postreise.plot.analyze_pg import AnalyzePG as apg
>>> from powersimdata.scenario.scenario import Scenario
>>> s = Scenario('398')
SCENARIO: test | EasternBase_2020_1

--> State
analyze
>>> zonelist = [
    'Maine',
    'New Hampshire',
    'Vermont',
    'Massachusetts',
    'Rhode Island',
    'Connecticut',
    'New York City',
    'Upstate New York',
    'New Jersey',
    'Pennsylvania Eastern',
    'Pennsylvania Western',
    'Delaware',
    'Maryland',
    'Virginia Mountains',
    'Virginia Tidewater',
    'North Carolina',
    'Western North Carolina',
    'South Carolina',
    'Georgia North',
    'Georgia South',
    'Florida Panhandle',
    'Florida North',
    'Florida South',
    'Alabama',
    'Mississippi',
    'Tennessee',
    'Kentucky',
    'West Virginia',
    'Ohio River',
    'Ohio Lake Erie',
    'Michigan Northern',
    'Michigan Southern',
    'Indiana',
    'Chicago North Illinois',
    'Illinois Downstate',
    'Wisconsin',
    'Minnesota Northern',
    'Minnesota Southern',
    'Iowa',
    'Missouri East',
    'Missouri West',
    'Arkansas',
    'Louisiana',
    'East Texas',
    'Texas Panhandle',
    'New Mexico Eastern',
    'Oklahoma',
    'Kansas',
    'Nebraska',
    'South Dakota',
    'North Dakota',
    'Montana Eastern',
    'Eastern'
    ]
>>> selected_resources = ['coal', 'dfo', 'geothermal', 'hydro', 'ng', 'nuclear', 'solar', 'wind']
>>> stack = apg(s, \
    ('2016-06-01-00', '2016-06-20-23', 'utc', 'H'), \
    zonelist, \
    selected_resources, \
    'stacked', normalize=False)
--> Loading PG
--> Loading grid
Loading bus
Loading plant
Loading heat_rate_curve
Loading gencost_before
Loading gencost_after
Loading branch
Loading dcline
Loading sub
Loading bus2sub
--> Loading demand
Multiply demand in Maine (#1) by 0.98
Multiply demand in New Hampshire (#2) by 0.98
Multiply demand in Vermont (#3) by 0.98
Multiply demand in Massachusetts (#4) by 0.98
Multiply demand in Rhode Island (#5) by 0.98
Multiply demand in Connecticut (#6) by 0.98
Multiply demand in New York City (#7) by 0.99
Multiply demand in Upstate New York (#8) by 0.99
Multiply demand in New Jersey (#9) by 1.04
Multiply demand in Pennsylvania Eastern (#10) by 1.02
Multiply demand in Pennsylvania Western (#11) by 1.02
Multiply demand in Delaware (#12) by 1.04
Multiply demand in Maryland (#13) by 1.01
Multiply demand in Virginia Mountains (#14) by 1.06
Multiply demand in Virginia Tidewater (#15) by 1.06
Multiply demand in North Carolina (#16) by 1.04
Multiply demand in Western North Carolina (#17) by 1.04
Multiply demand in South Carolina (#18) by 1.04
Multiply demand in Georgia North (#19) by 1.01
Multiply demand in Georgia South (#20) by 1.01
Multiply demand in Florida Panhandle (#21) by 1.04
Multiply demand in Florida North (#22) by 1.04
Multiply demand in Florida South (#23) by 1.04
Multiply demand in Alabama (#24) by 1.01
Multiply demand in Mississippi (#25) by 1.06
Multiply demand in Tennessee (#26) by 1.01
Multiply demand in Kentucky (#27) by 1.04
Multiply demand in West Virginia (#28) by 1.04
Multiply demand in Ohio River (#29) by 1.00
Multiply demand in Ohio Lake Erie (#30) by 1.00
Multiply demand in Michigan Northern (#31) by 1.02
Multiply demand in Michigan Southern (#32) by 1.02
Multiply demand in Indiana (#33) by 1.05
Multiply demand in Chicago North Illinois (#34) by 1.02
Multiply demand in Illinois Downstate (#35) by 1.02
Multiply demand in Wisconsin (#36) by 1.04
Multiply demand in Minnesota Northern (#37) by 1.03
Multiply demand in Minnesota Southern (#38) by 1.03
Multiply demand in Iowa (#39) by 1.06
Multiply demand in Missouri East (#40) by 1.04
Multiply demand in Missouri West (#41) by 1.04
Multiply demand in Arkansas (#42) by 1.04
Multiply demand in Louisiana (#43) by 1.02
Multiply demand in East Texas (#44) by 1.07
Multiply demand in Texas Panhandle (#45) by 1.07
Multiply demand in New Mexico Eastern (#46) by 1.02
Multiply demand in Oklahoma (#47) by 1.04
Multiply demand in Kansas (#48) by 1.02
Multiply demand in Nebraska (#49) by 1.02
Multiply demand in South Dakota (#50) by 1.07
Multiply demand in North Dakota (#51) by 1.06
Multiply demand in Montana Eastern (#52) by 1.05
--> Loading solar
--> Loading wind
--> Loading hydro

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-7-8f3dae0f2635> in <module>
      3             zonelist,
      4             selected_resources,
----> 5             'stacked', normalize=False)
      6 stack.get_plot(save=False)
      7 data_stack = stack.get_data()

c:\python37\lib\site-packages\postreise\plot\analyze_pg.py in __init__(self, scenario, time, zones, resources, kind, normalize, seed)
    137 
    138         if kind == 'stacked':
--> 139             self._do_stacked(time[0], time[1], time[2])
    140         elif kind == 'comp':
    141             self._do_comp(time[0], time[1], time[2])

c:\python37\lib\site-packages\postreise\plot\analyze_pg.py in _do_stacked(self, start_date, end_date, tz)
    425             self.tz = self.zone2time[z] if tz == 'local' else tz
    426             self._set_date_range(start_date, end_date)
--> 427             self.data.append(self._get_stacked(z))
    428 
    429     def _get_stacked(self, zone):

c:\python37\lib\site-packages\postreise\plot\analyze_pg.py in _get_stacked(self, zone)
    515             ax = pg_stack[list(type2label.keys())].tz_localize(None).rename(
    516                 columns=type2label).plot.area(
--> 517                 color=type2color, alpha=0.7, ax=ax)
    518 
    519             demand.tz_localize(None).plot(color='red', lw=4, ax=ax)

c:\python37\lib\site-packages\pandas\plotting\_core.py in area(self, x, y, **kwargs)
   1306             >>> ax = df.plot.area(x='day')
   1307         """
-> 1308         return self(kind="area", x=x, y=y, **kwargs)
   1309 
   1310     def pie(self, **kwargs):

c:\python37\lib\site-packages\pandas\plotting\_core.py in __call__(self, *args, **kwargs)
    792                     data.columns = label_name
    793 
--> 794         return plot_backend.plot(data, kind=kind, **kwargs)
    795 
    796     def line(self, x=None, y=None, **kwargs):

c:\python37\lib\site-packages\pandas\plotting\_matplotlib\__init__.py in plot(data, kind, **kwargs)
     60             kwargs["ax"] = getattr(ax, "left_ax", ax)
     61     plot_obj = PLOT_CLASSES[kind](data, **kwargs)
---> 62     plot_obj.generate()
     63     plot_obj.draw()
     64     return plot_obj.result

c:\python37\lib\site-packages\pandas\plotting\_matplotlib\core.py in generate(self)
    279         self._compute_plot_data()
    280         self._setup_subplots()
--> 281         self._make_plot()
    282         self._add_table()
    283         self._make_legend()

c:\python37\lib\site-packages\pandas\plotting\_matplotlib\core.py in _make_plot(self)
   1077                 stacking_id=stacking_id,
   1078                 is_errorbar=is_errorbar,
-> 1079                 **kwds
   1080             )
   1081             self._add_legend_handle(newlines[0], label, index=i)

c:\python37\lib\site-packages\pandas\plotting\_matplotlib\core.py in _ts_plot(cls, ax, x, data, style, **kwds)
   1118         ax._plot_data.append((data, cls._kind, kwds))
   1119 
-> 1120         lines = cls._plot(ax, data.index, data.values, style=style, **kwds)
   1121         # set date formatter, locators and rescale limits
   1122         format_dateaxis(ax, ax.freq, data.index)

c:\python37\lib\site-packages\pandas\plotting\_matplotlib\core.py in _plot(cls, ax, x, y, style, column_num, stacking_id, is_errorbar, **kwds)
   1233         if column_num == 0:
   1234             cls._initialize_stacker(ax, stacking_id, len(y))
-> 1235         y_values = cls._get_stacked_values(ax, stacking_id, y, kwds["label"])
   1236 
   1237         # need to remove label, because subplots uses mpl legend as it is

c:\python37\lib\site-packages\pandas\plotting\_matplotlib\core.py in _get_stacked_values(cls, ax, stacking_id, values, label)
   1156             "When stacked is True, each column must be either "
   1157             "all positive or negative."
-> 1158             "{0} contains both positive and negative values".format(label)
   1159         )
   1160 

ValueError: When stacked is True, each column must be either all positive or negative.Solar Curtailment contains both positive and negative values

[rouille]

Let’s clip the negative values to 0 as you propose. Do you want me to do it or you are already working on it?

[danielolsen]

I'm not working on it, I just wanted to document it, since I've run into it a couple of times.