Merge pull request #173 from Breakthrough-Energy/ben/curtailment

Extend curtailment calculation to areas

README.md

@@ -21,13 +21,13 @@ To carry out transmission congestion analyses per scenario:
 2. calculate congestion statistics;
 3. display the data.
 
-The ***[utilization_demo.ipynb][utilization]*** notebook shows the steps for 
-downloading, calculating and plotting the various statistics. Note that since 
+The ***[utilization_demo.ipynb][utilization]*** notebook shows the steps for
+downloading, calculating and plotting the various statistics. Note that since
 the plot outputs are meant to be interactive, they may not render on GitHub.
 
 ### B. Transmission Congestion (Surplus) Analysis
 The congestion surplus for each hour can be calculated by calling
-```
+```python
 postreise.analyze.transmission.congestion.calculate_congestion_surplus(scenario)
 ```
 where `scenario` is a powersimdata.scenario.scenario.Scenario object in Analyze
@@ -36,14 +36,14 @@ state.
 ### C. Carbon Analysis
 The hourly CO<sub>2</sub> emissions from a scenario may be analyzed by calling
 
-```
+```python
 postreise.analyze.generation.carbon.generate_carbon_stats(scenario)
 ```
-where `scenario` is a powersimdata.scenario.scenario.Scenario instance in the 
+where `scenario` is a powersimdata.scenario.scenario.Scenario instance in the
 analyze state.
 
 The resulting data frame can be summed by generator type and bus by calling
-```
+```python
 postreise.analyze.generation.carbon.summarize_carbon_by_bus(carbon, plant)
 ```
 where `carbon` is a pandas.DataFrame as returned by `generate_carbon_stats` and
@@ -52,51 +52,78 @@ where `carbon` is a pandas.DataFrame as returned by `generate_carbon_stats` and
 ### D. Curtailment Analysis
 The level of curtailment for a Scenario may be calculated in several ways.
 
+
+
 #### I. Calculating Time Series
 To calculate the time-series curtailment for each solar and wind generator, call
+```python
+from postreise.analyze.generation.curtailment import calculate_curtailment_time_series
+calculate_curtailment_time_series(scenario)
 ```
-postreise.analyze.generation.curtailment.calculate_curtailment_time_series(scenario)
+where `scenario` is a `powersimdata.scenario.scenario.Scenario` instance in the
+analyze state. If you call:
+```python
+postreise.analyze.generation.curtailment import calculate_curtailment_time_series_by_resources
+calculate_curtailment_time_series_by_resources(scenario, resources={"solar", "wind"})
 ```
-where `scenario` is a powersimdata.scenario.scenario.Scenario instance in the 
-analyze state. To calculate the curtailment just for wind or solar, call
-```
-postreise.analyze.generation.curtailment.calculate_curtailment_time_series(scenario, resources={'wind'})
-```
-or
+you will obtain a dictionary where the keys are solar and wind and the values are
+the curtailment time-series for the associated resource.
+
+You can also get the curtailment time-series by areas by calling:
+```python
+from postreise.analyze.generation.curtailment import calculate_curtailment_time_series_by_areas
+calculate_curtailment_time_series_by_areas(scenario,
+  areas={"interconnect": {"Western", "Texas"},
+         "state": {"Washington", "California", "Idaho"},
+         "loadzone": {"Bay Area", "El Paso", "Far West"}})
 ```
-postreise.analyze.generation.curtailment.calculate_curtailment_time_series(scenario, resources={'solar'})
+this returns a dictionary where the keys are the areas and the values are the
+curtailment time-series for solar and wind in the associated area.
+
+Finally, you can group the curtailment time-series by areas and resources as follows:
+```python
+postreise.analyze.generation.curtailment import calculate_curtailment_time_series_by_areas_and_resources
+calculate_curtailment_time_series_by_areas_and_resources(scenario,
+  areas={"interconnect": {"Western", "Texas"},
+         "state": {"Washington", "California", "Idaho"},
+         "loadzone": {"Bay Area", "El Paso", "Far West"}},
+  resources={"solar", "wind"})
 ```
+in that case the it returns a dictionary where the keys are the areas and the values
+are a dictionary where the keys are the resources and the values the curtailment
+time-series for the associated area-resource pair.
+
+
 
 #### II. Summarizing Time Series: Plant => Bus/Location
 A curtailment data frame with plants as columns can be further summarized by bus
 or by location (substation) with:
-```
-postreise.analyze.generation.curtailment.summarize_curtailment_by_bus(curtailment, grid)
+```python
+postreise.analyze.generation.curtailment.summarize_curtailment_by_bus(scenario)
 ```
 or
-
-```
-postreise.analyze.generation.curtailment.summarize_curtailment_by_location(curtailment, grid)
+```python
+postreise.analyze.generation.curtailment.summarize_curtailment_by_location(scenario)
 ```
 
-where `curtailment` is a pandas.DataFrame as returned by
-`calculate_curtailment_time_series` and `grid` is a
-powersimdata.input.grid.Grid instance.
+
 
 #### III. Calculating Annual Curtailment Percentage
 An annual average curtailment value can be found for all wind/solar plants with
 
-```
+```python
 postreise.analyze.generation.curtailment.calculate_curtailment_percentage(scenario)
 ```
-where `scenario` is a powersimdata.scenario.scenario.Scenario instance in the 
+where `scenario` is a `powersimdata.scenario.scenario.Scenario` instance in the
 analyze state. To calculate the average curtailment just for wind or solar, call
-```
-postreise.analyze.generation.curtailment.calculate_curtailment_percentage(scenario, resources={'wind'})
+```python
+from postreise.analyze.generation.curtailment import calculate_curtailment_percentage
+calculate_curtailment_percentage(scenario, resources={'wind'})
 ```
 or
-```
-postreise.analyze.generation.curtailment.calculate_curtailment_percentage(scenario, resources={'solar'})
+```python
+from ostreise.analyze.generation.curtailment import calculate_curtailment_percentage
+calculate_curtailment_percentage(scenario, resources={'solar'})
 ```
 
 
@@ -118,7 +145,7 @@ the same zone and using the same resource.
 * Calculate the capacity factor of one resource in one zone and show result
 using box plots.
 * Map the shadowprice and congested branches for a given hour
-* Tornado plot: Horizontal bar chart styled to show both positive and negative 
+* Tornado plot: Horizontal bar chart styled to show both positive and negative
 values cleanly.
 
 Check out the notebooks within the [demo][plot_notebooks] folder.
@@ -129,14 +156,14 @@ Check out the notebooks within the [demo][plot_notebooks] folder.
 
 The `plot_carbon_map` module is used to plot carbon emissions on a map.
 There are two ways it can be used:
-* Map carbon emissions per bus, size scaled to emissions quantity (tons) and 
+* Map carbon emissions per bus, size scaled to emissions quantity (tons) and
 color coded by fuel type.
 * Map carbon emissions per bus for two scenarios and compare.
-Comparison map color codes by increase vs. decrease from first to second 
+Comparison map color codes by increase vs. decrease from first to second
 scenario analyzed.
 
-The `plot_carbon_bar` module is used to make barcharts comparing carbon 
+The `plot_carbon_bar` module is used to make barcharts comparing carbon
 emissions of two scenarios.
 
-The `projection_helpers.py` module contains helper functions such as 
+The `projection_helpers.py` module contains helper functions such as
 re-projection, necessary for mapping.

postreise/analyze/check.py

@@ -1,21 +1,15 @@
 import datetime
+
 import numpy as np
 import pandas as pd
-
-from powersimdata.scenario.scenario import Scenario
-from powersimdata.scenario.analyze import Analyze
 from powersimdata.input.grid import Grid
+from powersimdata.network.usa_tamu.constants import zones
 from powersimdata.network.usa_tamu.constants.plants import (
     all_resources,
     renewable_resources,
 )
-from powersimdata.network.usa_tamu.constants.zones import (
-    loadzone,
-    state2loadzone,
-    abv2loadzone,
-    interconnect2loadzone,
-    abv2state,
-)
+from powersimdata.scenario.analyze import Analyze
+from powersimdata.scenario.scenario import Scenario
 
 
 def _check_data_frame(df, label):
@@ -85,7 +79,8 @@ def _check_areas_and_format(areas):
         state name(s)/abbreviation(s) or interconnect(s).
     :raises TypeError: if areas is not a list/tuple/set of str.
     :raises ValueError: if areas is empty or not valid.
-    :return: (*set*) -- areas as a set.
+    :return: (*set*) -- areas as a set. State abbreviations are converted to state
+        names.
     """
     if isinstance(areas, str):
         areas = {areas}
@@ -97,20 +92,15 @@ def _check_areas_and_format(areas):
         raise TypeError("areas must be a str or a list/tuple/set of str")
     if len(areas) == 0:
         raise ValueError("areas must be non-empty")
-    all_areas = (
-        loadzone
-        | set(abv2loadzone.keys())
-        | set(state2loadzone.keys())
-        | set(interconnect2loadzone.keys())
-    )
+    all_areas = zones.loadzone | zones.abv | zones.state | zones.interconnect
     if not areas <= all_areas:
         diff = areas - all_areas
         raise ValueError("invalid area(s): %s" % " | ".join(diff))
 
-    abv_in_areas = [z for z in areas if z in abv2state.keys()]
+    abv_in_areas = [z for z in areas if z in zones.abv]
     for a in abv_in_areas:
         areas.remove(a)
-        areas.add(abv2state[a])
+        areas.add(zones.abv2state[a])
 
     return areas
 
@@ -154,28 +144,90 @@ def _check_resources_are_renewable_and_format(resources):
     return resources
 
 
-def _check_resources_are_in_grid(resources, grid):
+def _check_areas_are_in_grid_and_format(areas, grid):
+    """Ensure that list of areas are in grid.
+
+    :param dict areas: keys are area types: '*loadzone*', '*state*' or '*interconnect*'.
+        Values are str/list/tuple/set of areas.
+    :param powersimdata.input.grid.Grid grid: Grid instance.
+    :return: (*dict*) -- modified areas dictionary. Keys are area types ('*loadzone*',
+        '*state*' or '*interconnect*'). State abbreviations, if present, are converted
+        to state names. Values are areas as a set.
+    :raises TypeError: if areas is not a dict or its keys are not str.
+    :raises ValueError: if area type is invalid, an area in not in grid or an
+        invalid loadzone/state/interconnect is passed.
+    """
+    _check_grid(grid)
+    if not isinstance(areas, dict):
+        raise TypeError("areas must be a dict")
+
+    areas_formatted = {}
+    for a in areas.keys():
+        if a in ["loadzone", "state", "interconnect"]:
+            areas_formatted[a] = set()
+
+    all_loadzones = set()
+    for k, v in areas.items():
+        if not isinstance(k, str):
+            raise TypeError("area type must be a str")
+        elif k == "interconnect":
+            interconnects = _check_areas_and_format(v)
+            for i in interconnects:
+                try:
+                    all_loadzones.update(zones.interconnect2loadzone[i])
+                except KeyError:
+                    raise ValueError("invalid interconnect: %s" % i)
+            areas_formatted["interconnect"].update(interconnects)
+        elif k == "state":
+            states = _check_areas_and_format(v)
+            for s in states:
+                try:
+                    all_loadzones.update(zones.state2loadzone[s])
+                except KeyError:
+                    raise ValueError("invalid state: %s" % s)
+            areas_formatted["state"].update(states)
+        elif k == "loadzone":
+            loadzones = _check_areas_and_format(v)
+            for l in loadzones:
+                if l not in zones.loadzone:
+                    raise ValueError("invalid load zone: %s" % l)
+            all_loadzones.update(loadzones)
+            areas_formatted["loadzone"].update(loadzones)
+        else:
+            raise ValueError("invalid area type")
+
+    valid_loadzones = set(grid.plant["zone_name"].unique())
+    if not all_loadzones <= valid_loadzones:
+        diff = all_loadzones - valid_loadzones
+        raise ValueError("%s not in in grid" % " | ".join(diff))
+
+    return areas_formatted
+
+
+def _check_resources_are_in_grid_and_format(resources, grid):
     """Ensure that resource(s) is represented in at least one generator in the grid
     used for the scenario.
 
     :param str/list/tuple/set resources: resource(s) to analyze.
     :param powersimdata.input.grid.Grid grid: a Grid instance.
+    :return: (*set*) -- resources as a set.
     :raises ValueError: if resources is not used in scenario.
     """
+    _check_grid(grid)
     resources = _check_resources_and_format(resources)
     valid_resources = set(grid.plant["type"].unique())
     if not resources <= valid_resources:
         diff = resources - valid_resources
         raise ValueError("%s not in in grid" % " | ".join(diff))
+    return resources
 
 
 def _check_plants_are_in_grid(plant_id, grid):
     """Ensure that list of plant id are in grid.
 
-    :param list/tuple/set plant_id: list of plant_id.
+    :param list/tuple/set plant_id: list of plant id.
     :param powersimdata.input.grid.Grid grid: Grid instance.
-    :raises TypeError: if plant_id is not a list of int or str and grid is not a Grid
-        object.
+    :raises TypeError: if plant_id is not a list of int or str.
     :raises ValueError: if plant id is not in network.
     """
     _check_grid(grid)
@@ -184,8 +236,6 @@ def _check_plants_are_in_grid(plant_id, grid):
         and all([isinstance(p, (int, str)) for p in plant_id])
     ):
         raise TypeError("plant_id must be a a list/tuple/set of int or str")
-    if not isinstance(grid, Grid):
-        raise TypeError("grid must be powersimdata.input.grid.Grid object")
     if not set([str(p) for p in plant_id]) <= set([str(i) for i in grid.plant.index]):
         raise ValueError("plant_id must be subset of plant index")
 
@@ -313,20 +363,3 @@ def _check_gencost(gencost):
     for c in coef_columns:
         if c not in gencost.columns:
             raise ValueError("gencost of order {0} must have column {1}".format(n, c))
-
-
-def _check_curtailment(curtailment, grid):
-    """Ensure that curtailment is a dict of data frames, and that each key is
-    represented in at least one generator in grid.
-
-    :param dict curtailment: curtailment data.
-    :param powersimdata.input.grid.Grid grid: a Grid object.
-    :raises TypeError: if curtailment is not a dict and values are not data frames.
-    """
-    if not isinstance(curtailment, dict):
-        raise TypeError("curtailment must be a dict")
-    _check_grid(grid)
-    resources = _check_resources_are_renewable_and_format(list(curtailment.keys()))
-    _check_resources_are_in_grid(resources, grid)
-    if not all([isinstance(v, pd.DataFrame) for v in curtailment.values()]):
-        raise TypeError("curtailment values must be a data frame")

postreise/analyze/generation/capacity_value.py

@@ -1,7 +1,6 @@
 from postreise.analyze.check import (
     _check_scenario_is_in_analyze_state,
-    _check_resources_and_format,
-    _check_resources_are_in_grid,
+    _check_resources_are_in_grid_and_format,
     _check_number_hours_to_analyze,
 )
 
@@ -18,8 +17,7 @@ def calculate_NLDC(scenario, resources, hours=100):
     """
     _check_scenario_is_in_analyze_state(scenario)
     grid = scenario.state.get_grid()
-    resources = _check_resources_and_format(resources)
-    _check_resources_are_in_grid(resources, grid)
+    resources = _check_resources_are_in_grid_and_format(resources, grid)
     _check_number_hours_to_analyze(scenario, hours)
 
     # Then calculate capacity value
@@ -46,8 +44,7 @@ def calculate_net_load_peak(scenario, resources, hours=100):
     """
     _check_scenario_is_in_analyze_state(scenario)
     grid = scenario.state.get_grid()
-    resources = _check_resources_and_format(resources)
-    _check_resources_are_in_grid(resources, grid)
+    resources = _check_resources_are_in_grid_and_format(resources, grid)
     _check_number_hours_to_analyze(scenario, hours)
 
     # Then calculate capacity value