Outputs w std fmt #182
Merged
Outputs w std fmt #182
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add new files for rebasing
bennettm
force-pushed
the
outputs_w_std_fmt
branch
from
23518ab to
9dce339
Compare
RuaridhMacd
approved these changes
Nov 16, 2023
RuaridhMacd
added a commit
that referenced
this pull request
Jan 18, 2024
* Create expression_manipulation * Fix expression_manipulation file title * Create expression_manipulation * Fix expression_manipulation file title * Correct zone values in h2_pipeline_flow.csv file (#184) Co-authored-by: Mary Bennett <[email protected]> * Auto test (#173) * configure highs solver * change solver option to highs * check results using compare_results * test script * Update Test-OneZone.yml checkout submodules. * Update and rename Test-OneZone.yml to TestHelper.yml 1. Add threezones, liquid and northsea case test 2. Add keywords ['#skip', 'CompatHelper'] to allow test bypassing to save unnecessary github action * Update TestHelper.yml Pair condition judgment. #skip. * add test package * drop output file * compare results * Update TestHelper.yml test PR * Update TestHelper.yml notify author about the comparison results * Update TestHelper.yml test when PR is opened to develop * Update TestHelper.yml fetch submodule recursively * Update TestHelper.yml add email sender from user * Update TestHelper.yml dump github context * Update TestHelper.yml Add PR comment instead of writing emails * Update TestHelper.yml permissions: pull-requests: write * Update TestHelper.yml * Update TestHelper.yml Update comment instead of creating a new one and format message in code block * compare to cached results folders * Update TestHelper.yml retrive cache for comparison * Create TestMainHelper.yml generate results cache for comparison * Add arch choice to Ubuntu --------- Co-authored-by: Mary Bennett <[email protected]> * Develop tmr (#172) * Time matching requirments 3 (#171) * Modified code to model hourly matching requirement Modified scripts to model hourly matching requirements * Fix parameter error in FRCC example - Up_time and Down_time for therm=1 generator was set to 0 in the FRCC_2021_houryl_1GW_base example case study. This led to an error in generating constraints for thermal_commit.jl - Added MaxCapReq = 0 as the default value in configure_settings.jl * Updated example to include MaxCapTag column Added MaxCapTag column to allow specifying those generators eligible for maximum capacity constraint * Modified input data for FRCC cases * Time matching requirement for electricity based H2 production Added a policy constraint on the HSC side to model time-matching requirements for H2 production. User input data: - Generators_data.csv: Define a column H2_TMR_XXX (e.g. H2_TMR_1, H2_TMR_2 ) and set qualifying power resources to 1 and rest of the power resources to zero - HSC_generation.csv: Define a column H2_TMR_XXX (e.g. H2_TMR_1, H2_TMR_2 ) to determine the H2 production resources for which the time matching requirement needs to be enforced. Features: - Time matching requirement for electricity based H2 production - Ability of resources used to meet time matching requirement to also contribute to energy share requirement on the power system side. * Update generate_model.jl to match main * Revert "Update generate_model.jl to match main" This reverts commit ac249b9. * Update branch with example input files * Add option to disallow excess sales from TMR resources to participate in ESR constraint Add a setup option that user can set to 0 or 1 to disallow or allow excess electricity sales from resources contracted for time-matching requirements for H2 production to be counted towards power sector energy share requirements. --------- Co-authored-by: dharik13 <[email protected]> * restoring submodule from main * minor modifications * made change to account for no CO2 Cap case. Passing in multistage setup variable expected by GenX * Added new ESR to TMR mapping functionality TMRs are now not added to all ESR, but only to relavent ESRs * Added if statement in case excess sales do not go to ESR, reordered generate model * CO2 cap exception handling 1. Changed generate_model.jl to skip CO2_cap if setup flag for CO2Cap =0 2. Change co2_cap_power_hsc.jl to skip CO2_cap.jl and co2_cap_hsc.jl if setup flag for CO2Cap and H2CO2Cap are both zero. * update configure highs * removing old version of configure_highs.jl * check out configure_highs.jl from main * added missing h2_pipeline expression back * reverting order of generate_model.jl * added ESR expression to generate model * fixed bugs with tmr * modified settings. Updated tmr to account for case when there is no TMRs in ESRs. * updated documentation, added example system * Update write_h2_tmr_prices.jl * updated documentation * Edited documentation and removed redundant example - Removed extra example - Edited documentation for time_matching_requirement.jl and other files where I found some errors * Uncommented instantiate package line in make.jl * Added GenX data documentation changes copied data_documentation from src/GenX/docs to DOLPHYN/src/docs. did not modify the make.jl to point this file, since this is already resolved in develop. * Docstring Changes for Correct Output * Update write_h2_elec_costs.jl We were attempting to multiply arrays of size: ZxT .* ZxT .* T Fixed by transposing the final array --------- Co-authored-by: dharik13 <[email protected]> Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: Ruaridh Macdonald <[email protected]> * Create expression_manipulation * Fix expression_manipulation file title * Create expression_manipulation * Fix expression_manipulation file title * Correct zone values in h2_pipeline_flow.csv file (#184) Co-authored-by: Mary Bennett <[email protected]> * Auto test (#173) * configure highs solver * change solver option to highs * check results using compare_results * test script * Update Test-OneZone.yml checkout submodules. * Update and rename Test-OneZone.yml to TestHelper.yml 1. Add threezones, liquid and northsea case test 2. Add keywords ['#skip', 'CompatHelper'] to allow test bypassing to save unnecessary github action * Update TestHelper.yml Pair condition judgment. #skip. * add test package * drop output file * compare results * Update TestHelper.yml test PR * Update TestHelper.yml notify author about the comparison results * Update TestHelper.yml test when PR is opened to develop * Update TestHelper.yml fetch submodule recursively * Update TestHelper.yml add email sender from user * Update TestHelper.yml dump github context * Update TestHelper.yml Add PR comment instead of writing emails * Update TestHelper.yml permissions: pull-requests: write * Update TestHelper.yml * Update TestHelper.yml Update comment instead of creating a new one and format message in code block * compare to cached results folders * Update TestHelper.yml retrive cache for comparison * Create TestMainHelper.yml generate results cache for comparison * Add arch choice to Ubuntu --------- Co-authored-by: Mary Bennett <[email protected]> * Develop tmr (#172) * Time matching requirments 3 (#171) * Modified code to model hourly matching requirement Modified scripts to model hourly matching requirements * Fix parameter error in FRCC example - Up_time and Down_time for therm=1 generator was set to 0 in the FRCC_2021_houryl_1GW_base example case study. This led to an error in generating constraints for thermal_commit.jl - Added MaxCapReq = 0 as the default value in configure_settings.jl * Updated example to include MaxCapTag column Added MaxCapTag column to allow specifying those generators eligible for maximum capacity constraint * Modified input data for FRCC cases * Time matching requirement for electricity based H2 production Added a policy constraint on the HSC side to model time-matching requirements for H2 production. User input data: - Generators_data.csv: Define a column H2_TMR_XXX (e.g. H2_TMR_1, H2_TMR_2 ) and set qualifying power resources to 1 and rest of the power resources to zero - HSC_generation.csv: Define a column H2_TMR_XXX (e.g. H2_TMR_1, H2_TMR_2 ) to determine the H2 production resources for which the time matching requirement needs to be enforced. Features: - Time matching requirement for electricity based H2 production - Ability of resources used to meet time matching requirement to also contribute to energy share requirement on the power system side. * Update generate_model.jl to match main * Revert "Update generate_model.jl to match main" This reverts commit ac249b9. * Update branch with example input files * Add option to disallow excess sales from TMR resources to participate in ESR constraint Add a setup option that user can set to 0 or 1 to disallow or allow excess electricity sales from resources contracted for time-matching requirements for H2 production to be counted towards power sector energy share requirements. --------- Co-authored-by: dharik13 <[email protected]> * restoring submodule from main * minor modifications * made change to account for no CO2 Cap case. Passing in multistage setup variable expected by GenX * Added new ESR to TMR mapping functionality TMRs are now not added to all ESR, but only to relavent ESRs * Added if statement in case excess sales do not go to ESR, reordered generate model * CO2 cap exception handling 1. Changed generate_model.jl to skip CO2_cap if setup flag for CO2Cap =0 2. Change co2_cap_power_hsc.jl to skip CO2_cap.jl and co2_cap_hsc.jl if setup flag for CO2Cap and H2CO2Cap are both zero. * update configure highs * removing old version of configure_highs.jl * check out configure_highs.jl from main * added missing h2_pipeline expression back * reverting order of generate_model.jl * added ESR expression to generate model * fixed bugs with tmr * modified settings. Updated tmr to account for case when there is no TMRs in ESRs. * updated documentation, added example system * Update write_h2_tmr_prices.jl * updated documentation * Edited documentation and removed redundant example - Removed extra example - Edited documentation for time_matching_requirement.jl and other files where I found some errors * Uncommented instantiate package line in make.jl * Added GenX data documentation changes copied data_documentation from src/GenX/docs to DOLPHYN/src/docs. did not modify the make.jl to point this file, since this is already resolved in develop. * Docstring Changes for Correct Output * Update write_h2_elec_costs.jl We were attempting to multiply arrays of size: ZxT .* ZxT .* T Fixed by transposing the final array --------- Co-authored-by: dharik13 <[email protected]> Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: Ruaridh Macdonald <[email protected]> * Housecleaning after rebasing from v0.2.2 main branch * refactor dolphyn for less memory and better performance (#175) * refactor dolphyn for less memory and better performance * Revert "compare results" This reverts commit b473e63. * Revert "add test package" This reverts commit 32cebae. * Revert "change solver from gurobi to highs" This reverts commit 54b0777. * Simplifying HSC output writing - Moved remaining indexes inside value() functions - Moved if-statements outside loops if they didn't need to be called - Started using insertcols!() to replace some for-loops on z=1:Z and t=1:T Testing so far using @benchmark shows 10-30% reduction in memory usage and 5-10% reduction in total model runtime for the SmallNewEngland and NorthSea2030. * Fix bug in write_h2_storage Previously got error if either set of indices was empty * Revert "Revert "compare results"" This reverts commit 31e5db5. * Revert "Revert "add test package"" This reverts commit 059fe70. * Revert "Revert "change solver from gurobi to highs"" This reverts commit d2763f8. --------- Co-authored-by: RuaridhMacd <[email protected]> * Outputs w std fmt (#182) * Standard Format for Output Files Add new files for rebasing * Update GenX exclusions to avoid conflicts --------- Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: RuaridhMacd <[email protected]> * Liquid fuels merge (#178) * Big intital push * minor changes * changed CO2 constraint structure modified generate model and run file * added comment * creating ccs rate read in function, adjusting co2 cap file * Added bar homogenus, removed check for CO2, added min no_commit gen, added startup costs by resource exp * Added green_h2_share_req, fixed emissions_hsc, reads in blue and green h2, fixed issues w ccs * fixed issues with ccs * Big intital push * minor changes * changed CO2 constraint structure modified generate model and run file * added comment * creating ccs rate read in function, adjusting co2 cap file * Added bar homogenus, removed check for CO2, added min no_commit gen, added startup costs by resource exp * Added green_h2_share_req, fixed emissions_hsc, reads in blue and green h2, fixed issues w ccs * fixed issues with ccs * emissions cap handling for power + csc without h2 * Debugging -Modified example - Updated project toml to include gurobi - Removed duplication in config settings - modified configure_gurobi - modified runtools to include csc - added choose csc output dir - modified write_csc_ouputs to match HSC structure - Fixed bugs in generate_model - fixed bugs in emissions.jl * fixed emissions constraint bug * modified case. Modified write outputs * Fix new bug in thermal_commit The behaviour of the sum() function was changed in a new version of JuMP. This has been patched in GenX so I've copied the change here till we decide to update all of GenX. I also corrected the load file of the CSC/ThreeZones example * CSC Documentations Fully completed documentations for CSC incode and doc. Did testing on CSC and updated H2 share policy. * Fix a bug when CSC is turned off * Fix to documentations * Update make.jl * Fix local liveserver does not update changes error * Add updated Project.toml * Fix bugs to CSC documentations * 203 liquid fuels hsc output writing files merge and optimization (#204) * Merge HSC output files from Liquid_Fuels to Liquid_Fuels_Merge * Add codes to reduce redundancy for optimization * Fixed bugs in write_h2_balance and write_h2_capacity * Bug fixes for h2 The code has mistakes that prevents it from running. I have made the necessary fixes to review and update the code as necessary. Some of the optimizations implemented that have resulted in errors. More specifically, I have reverted the following files to their older versions (write_g2p_capcity.jl, write_h2_balance_dual.jl, write_p_g2p.jl, write_h2_g2p.jl). As a result, I recommend that once I merge the PR (and merge to develop), you can create a separate PR for optimizing the write outputs files. --------- Co-authored-by: shaker <[email protected]> * bug fix in h2_balance, removed repeated lines in generate_model * flag bug fix * Added error handling for no csc case settings loading * fixed typo * typo fix * updated example to include ccs_rate * updated single model, removed incorrect const * updated default settings * updated settings * typo * updated 3 zone inputs * updated north sea example * updates liquid fuel example --------- Co-authored-by: Ruaridh Macdonald <[email protected]> Co-authored-by: Junwenlaw <[email protected]> Co-authored-by: Mary Bennett <[email protected]> * Syn fuels merge new (#208) * intial push * push to debug liquid fuels - Updated run file - updated liquid fuel settigns file - updated default settings - Updated Generate Model - Fixed typos in HSC output files - changed naming conventions in liquid fuel demand - added choose lf output dir - updated emissions file * updated run file and settings * ovewrote capacity reserve margin to account for h2 * added logfile defaults, added example systems, fixed typo in liquid_fuel_demand_module * Updated Examples * default parameter scale to 0 when liquid fuels is on * Testing and documentations done * Set parent directory of docs in development mode * Add minor corrections to the documentation files * fixed bug in liquid fuel demand, modified configure settings, modified examples. --------- Co-authored-by: Junwenlaw <[email protected]> Co-authored-by: Mary Bennett <[email protected]> * Fix solver configuration process, add example using Gurobi, swap all to HiGHS Previously we were silently loading Gurobi, CPLEX, etc if they were installed but not loaded. This meant we didn't have to ask the user to change the solver manually, but led to unusual behaviour (like it never working the first time it's run but then being fine afterwards) and is probably bad practice. No the user has to define the solver when they create the runfile. It means the "solver" setting is a bit redundant and perhaps misleading. We should decide if we want to keep it or try a different approach. * Remove code for LiveServer This crashes doc creation and deployment on GitHub Actions * Clean-up unused files * Update Project.toml * Clean up examples and add summary files to .gitignore --------- Co-authored-by: Ruaridh Macdonald <[email protected]> Co-authored-by: bennettm <[email protected]> Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: Yuheng Zhang <[email protected]> Co-authored-by: dharik13 <[email protected]> Co-authored-by: bennettm <[email protected]> Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: Junwenlaw <[email protected]>
RuaridhMacd
added a commit
that referenced
this pull request
Feb 2, 2024
* merge csc and lfsc modules into main (#209) * Create expression_manipulation * Fix expression_manipulation file title * Create expression_manipulation * Fix expression_manipulation file title * Correct zone values in h2_pipeline_flow.csv file (#184) Co-authored-by: Mary Bennett <[email protected]> * Auto test (#173) * configure highs solver * change solver option to highs * check results using compare_results * test script * Update Test-OneZone.yml checkout submodules. * Update and rename Test-OneZone.yml to TestHelper.yml 1. Add threezones, liquid and northsea case test 2. Add keywords ['#skip', 'CompatHelper'] to allow test bypassing to save unnecessary github action * Update TestHelper.yml Pair condition judgment. #skip. * add test package * drop output file * compare results * Update TestHelper.yml test PR * Update TestHelper.yml notify author about the comparison results * Update TestHelper.yml test when PR is opened to develop * Update TestHelper.yml fetch submodule recursively * Update TestHelper.yml add email sender from user * Update TestHelper.yml dump github context * Update TestHelper.yml Add PR comment instead of writing emails * Update TestHelper.yml permissions: pull-requests: write * Update TestHelper.yml * Update TestHelper.yml Update comment instead of creating a new one and format message in code block * compare to cached results folders * Update TestHelper.yml retrive cache for comparison * Create TestMainHelper.yml generate results cache for comparison * Add arch choice to Ubuntu --------- Co-authored-by: Mary Bennett <[email protected]> * Develop tmr (#172) * Time matching requirments 3 (#171) * Modified code to model hourly matching requirement Modified scripts to model hourly matching requirements * Fix parameter error in FRCC example - Up_time and Down_time for therm=1 generator was set to 0 in the FRCC_2021_houryl_1GW_base example case study. This led to an error in generating constraints for thermal_commit.jl - Added MaxCapReq = 0 as the default value in configure_settings.jl * Updated example to include MaxCapTag column Added MaxCapTag column to allow specifying those generators eligible for maximum capacity constraint * Modified input data for FRCC cases * Time matching requirement for electricity based H2 production Added a policy constraint on the HSC side to model time-matching requirements for H2 production. User input data: - Generators_data.csv: Define a column H2_TMR_XXX (e.g. H2_TMR_1, H2_TMR_2 ) and set qualifying power resources to 1 and rest of the power resources to zero - HSC_generation.csv: Define a column H2_TMR_XXX (e.g. H2_TMR_1, H2_TMR_2 ) to determine the H2 production resources for which the time matching requirement needs to be enforced. Features: - Time matching requirement for electricity based H2 production - Ability of resources used to meet time matching requirement to also contribute to energy share requirement on the power system side. * Update generate_model.jl to match main * Revert "Update generate_model.jl to match main" This reverts commit ac249b9. * Update branch with example input files * Add option to disallow excess sales from TMR resources to participate in ESR constraint Add a setup option that user can set to 0 or 1 to disallow or allow excess electricity sales from resources contracted for time-matching requirements for H2 production to be counted towards power sector energy share requirements. --------- Co-authored-by: dharik13 <[email protected]> * restoring submodule from main * minor modifications * made change to account for no CO2 Cap case. Passing in multistage setup variable expected by GenX * Added new ESR to TMR mapping functionality TMRs are now not added to all ESR, but only to relavent ESRs * Added if statement in case excess sales do not go to ESR, reordered generate model * CO2 cap exception handling 1. Changed generate_model.jl to skip CO2_cap if setup flag for CO2Cap =0 2. Change co2_cap_power_hsc.jl to skip CO2_cap.jl and co2_cap_hsc.jl if setup flag for CO2Cap and H2CO2Cap are both zero. * update configure highs * removing old version of configure_highs.jl * check out configure_highs.jl from main * added missing h2_pipeline expression back * reverting order of generate_model.jl * added ESR expression to generate model * fixed bugs with tmr * modified settings. Updated tmr to account for case when there is no TMRs in ESRs. * updated documentation, added example system * Update write_h2_tmr_prices.jl * updated documentation * Edited documentation and removed redundant example - Removed extra example - Edited documentation for time_matching_requirement.jl and other files where I found some errors * Uncommented instantiate package line in make.jl * Added GenX data documentation changes copied data_documentation from src/GenX/docs to DOLPHYN/src/docs. did not modify the make.jl to point this file, since this is already resolved in develop. * Docstring Changes for Correct Output * Update write_h2_elec_costs.jl We were attempting to multiply arrays of size: ZxT .* ZxT .* T Fixed by transposing the final array --------- Co-authored-by: dharik13 <[email protected]> Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: Ruaridh Macdonald <[email protected]> * Create expression_manipulation * Fix expression_manipulation file title * Create expression_manipulation * Fix expression_manipulation file title * Correct zone values in h2_pipeline_flow.csv file (#184) Co-authored-by: Mary Bennett <[email protected]> * Auto test (#173) * configure highs solver * change solver option to highs * check results using compare_results * test script * Update Test-OneZone.yml checkout submodules. * Update and rename Test-OneZone.yml to TestHelper.yml 1. Add threezones, liquid and northsea case test 2. Add keywords ['#skip', 'CompatHelper'] to allow test bypassing to save unnecessary github action * Update TestHelper.yml Pair condition judgment. #skip. * add test package * drop output file * compare results * Update TestHelper.yml test PR * Update TestHelper.yml notify author about the comparison results * Update TestHelper.yml test when PR is opened to develop * Update TestHelper.yml fetch submodule recursively * Update TestHelper.yml add email sender from user * Update TestHelper.yml dump github context * Update TestHelper.yml Add PR comment instead of writing emails * Update TestHelper.yml permissions: pull-requests: write * Update TestHelper.yml * Update TestHelper.yml Update comment instead of creating a new one and format message in code block * compare to cached results folders * Update TestHelper.yml retrive cache for comparison * Create TestMainHelper.yml generate results cache for comparison * Add arch choice to Ubuntu --------- Co-authored-by: Mary Bennett <[email protected]> * Develop tmr (#172) * Time matching requirments 3 (#171) * Modified code to model hourly matching requirement Modified scripts to model hourly matching requirements * Fix parameter error in FRCC example - Up_time and Down_time for therm=1 generator was set to 0 in the FRCC_2021_houryl_1GW_base example case study. This led to an error in generating constraints for thermal_commit.jl - Added MaxCapReq = 0 as the default value in configure_settings.jl * Updated example to include MaxCapTag column Added MaxCapTag column to allow specifying those generators eligible for maximum capacity constraint * Modified input data for FRCC cases * Time matching requirement for electricity based H2 production Added a policy constraint on the HSC side to model time-matching requirements for H2 production. User input data: - Generators_data.csv: Define a column H2_TMR_XXX (e.g. H2_TMR_1, H2_TMR_2 ) and set qualifying power resources to 1 and rest of the power resources to zero - HSC_generation.csv: Define a column H2_TMR_XXX (e.g. H2_TMR_1, H2_TMR_2 ) to determine the H2 production resources for which the time matching requirement needs to be enforced. Features: - Time matching requirement for electricity based H2 production - Ability of resources used to meet time matching requirement to also contribute to energy share requirement on the power system side. * Update generate_model.jl to match main * Revert "Update generate_model.jl to match main" This reverts commit ac249b9. * Update branch with example input files * Add option to disallow excess sales from TMR resources to participate in ESR constraint Add a setup option that user can set to 0 or 1 to disallow or allow excess electricity sales from resources contracted for time-matching requirements for H2 production to be counted towards power sector energy share requirements. --------- Co-authored-by: dharik13 <[email protected]> * restoring submodule from main * minor modifications * made change to account for no CO2 Cap case. Passing in multistage setup variable expected by GenX * Added new ESR to TMR mapping functionality TMRs are now not added to all ESR, but only to relavent ESRs * Added if statement in case excess sales do not go to ESR, reordered generate model * CO2 cap exception handling 1. Changed generate_model.jl to skip CO2_cap if setup flag for CO2Cap =0 2. Change co2_cap_power_hsc.jl to skip CO2_cap.jl and co2_cap_hsc.jl if setup flag for CO2Cap and H2CO2Cap are both zero. * update configure highs * removing old version of configure_highs.jl * check out configure_highs.jl from main * added missing h2_pipeline expression back * reverting order of generate_model.jl * added ESR expression to generate model * fixed bugs with tmr * modified settings. Updated tmr to account for case when there is no TMRs in ESRs. * updated documentation, added example system * Update write_h2_tmr_prices.jl * updated documentation * Edited documentation and removed redundant example - Removed extra example - Edited documentation for time_matching_requirement.jl and other files where I found some errors * Uncommented instantiate package line in make.jl * Added GenX data documentation changes copied data_documentation from src/GenX/docs to DOLPHYN/src/docs. did not modify the make.jl to point this file, since this is already resolved in develop. * Docstring Changes for Correct Output * Update write_h2_elec_costs.jl We were attempting to multiply arrays of size: ZxT .* ZxT .* T Fixed by transposing the final array --------- Co-authored-by: dharik13 <[email protected]> Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: Ruaridh Macdonald <[email protected]> * Housecleaning after rebasing from v0.2.2 main branch * refactor dolphyn for less memory and better performance (#175) * refactor dolphyn for less memory and better performance * Revert "compare results" This reverts commit b473e63. * Revert "add test package" This reverts commit 32cebae. * Revert "change solver from gurobi to highs" This reverts commit 54b0777. * Simplifying HSC output writing - Moved remaining indexes inside value() functions - Moved if-statements outside loops if they didn't need to be called - Started using insertcols!() to replace some for-loops on z=1:Z and t=1:T Testing so far using @benchmark shows 10-30% reduction in memory usage and 5-10% reduction in total model runtime for the SmallNewEngland and NorthSea2030. * Fix bug in write_h2_storage Previously got error if either set of indices was empty * Revert "Revert "compare results"" This reverts commit 31e5db5. * Revert "Revert "add test package"" This reverts commit 059fe70. * Revert "Revert "change solver from gurobi to highs"" This reverts commit d2763f8. --------- Co-authored-by: RuaridhMacd <[email protected]> * Outputs w std fmt (#182) * Standard Format for Output Files Add new files for rebasing * Update GenX exclusions to avoid conflicts --------- Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: RuaridhMacd <[email protected]> * Liquid fuels merge (#178) * Big intital push * minor changes * changed CO2 constraint structure modified generate model and run file * added comment * creating ccs rate read in function, adjusting co2 cap file * Added bar homogenus, removed check for CO2, added min no_commit gen, added startup costs by resource exp * Added green_h2_share_req, fixed emissions_hsc, reads in blue and green h2, fixed issues w ccs * fixed issues with ccs * Big intital push * minor changes * changed CO2 constraint structure modified generate model and run file * added comment * creating ccs rate read in function, adjusting co2 cap file * Added bar homogenus, removed check for CO2, added min no_commit gen, added startup costs by resource exp * Added green_h2_share_req, fixed emissions_hsc, reads in blue and green h2, fixed issues w ccs * fixed issues with ccs * emissions cap handling for power + csc without h2 * Debugging -Modified example - Updated project toml to include gurobi - Removed duplication in config settings - modified configure_gurobi - modified runtools to include csc - added choose csc output dir - modified write_csc_ouputs to match HSC structure - Fixed bugs in generate_model - fixed bugs in emissions.jl * fixed emissions constraint bug * modified case. Modified write outputs * Fix new bug in thermal_commit The behaviour of the sum() function was changed in a new version of JuMP. This has been patched in GenX so I've copied the change here till we decide to update all of GenX. I also corrected the load file of the CSC/ThreeZones example * CSC Documentations Fully completed documentations for CSC incode and doc. Did testing on CSC and updated H2 share policy. * Fix a bug when CSC is turned off * Fix to documentations * Update make.jl * Fix local liveserver does not update changes error * Add updated Project.toml * Fix bugs to CSC documentations * 203 liquid fuels hsc output writing files merge and optimization (#204) * Merge HSC output files from Liquid_Fuels to Liquid_Fuels_Merge * Add codes to reduce redundancy for optimization * Fixed bugs in write_h2_balance and write_h2_capacity * Bug fixes for h2 The code has mistakes that prevents it from running. I have made the necessary fixes to review and update the code as necessary. Some of the optimizations implemented that have resulted in errors. More specifically, I have reverted the following files to their older versions (write_g2p_capcity.jl, write_h2_balance_dual.jl, write_p_g2p.jl, write_h2_g2p.jl). As a result, I recommend that once I merge the PR (and merge to develop), you can create a separate PR for optimizing the write outputs files. --------- Co-authored-by: shaker <[email protected]> * bug fix in h2_balance, removed repeated lines in generate_model * flag bug fix * Added error handling for no csc case settings loading * fixed typo * typo fix * updated example to include ccs_rate * updated single model, removed incorrect const * updated default settings * updated settings * typo * updated 3 zone inputs * updated north sea example * updates liquid fuel example --------- Co-authored-by: Ruaridh Macdonald <[email protected]> Co-authored-by: Junwenlaw <[email protected]> Co-authored-by: Mary Bennett <[email protected]> * Syn fuels merge new (#208) * intial push * push to debug liquid fuels - Updated run file - updated liquid fuel settigns file - updated default settings - Updated Generate Model - Fixed typos in HSC output files - changed naming conventions in liquid fuel demand - added choose lf output dir - updated emissions file * updated run file and settings * ovewrote capacity reserve margin to account for h2 * added logfile defaults, added example systems, fixed typo in liquid_fuel_demand_module * Updated Examples * default parameter scale to 0 when liquid fuels is on * Testing and documentations done * Set parent directory of docs in development mode * Add minor corrections to the documentation files * fixed bug in liquid fuel demand, modified configure settings, modified examples. --------- Co-authored-by: Junwenlaw <[email protected]> Co-authored-by: Mary Bennett <[email protected]> * Fix solver configuration process, add example using Gurobi, swap all to HiGHS Previously we were silently loading Gurobi, CPLEX, etc if they were installed but not loaded. This meant we didn't have to ask the user to change the solver manually, but led to unusual behaviour (like it never working the first time it's run but then being fine afterwards) and is probably bad practice. No the user has to define the solver when they create the runfile. It means the "solver" setting is a bit redundant and perhaps misleading. We should decide if we want to keep it or try a different approach. * Remove code for LiveServer This crashes doc creation and deployment on GitHub Actions * Clean-up unused files * Update Project.toml * Clean up examples and add summary files to .gitignore --------- Co-authored-by: Ruaridh Macdonald <[email protected]> Co-authored-by: bennettm <[email protected]> Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: Yuheng Zhang <[email protected]> Co-authored-by: dharik13 <[email protected]> Co-authored-by: bennettm <[email protected]> Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: Junwenlaw <[email protected]> * Update Project.toml to version 0.2.3 --------- Co-authored-by: shakesy94 <[email protected]> Co-authored-by: bennettm <[email protected]> Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: Yuheng Zhang <[email protected]> Co-authored-by: dharik13 <[email protected]> Co-authored-by: bennettm <[email protected]> Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: Junwenlaw <[email protected]>
RuaridhMacd
added a commit
that referenced
this pull request
May 1, 2024
* Standard Format for Output Files Add new files for rebasing * Update GenX exclusions to avoid conflicts --------- Co-authored-by: Mary Bennett <[email protected]> Co-authored-by: RuaridhMacd <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Output files for GenX and HSC have been modified to have the standard format in their CSV results files. The standard format follows the rule that users generally see in this order starting from the top row: Resources, Zone, AnnualSum/AnnualMean, and other attributes. It also depends on the content of the file so that some files may not need to have exactly this format.
Here is an example of what a standard format looks like taken from power.csv:
Any GenX output files that have been changed for the standard format purpose have been added to src/GenX_extensions. Documentation is not up to date with these new additions to GenX_extensions.
These are the files that have changes:
src/GenX_extensions/write_capacity.jl
src/GenX_extensions/write_capacityfactor.jl
src/GenX_extensions/write_charging_cost.jl
src/GenX_extensions/write_energy_revenue.jl
src/GenX_extensions/write_net_revenue.jl
src/GenX_extensions/write_price.jl
src/GenX_extensions/write_reliability.jl
src/GenX_extensions/write_storage.jl
src/GenX_extensions/write_storagedual.jl
src/GenX_extensions/write_subsidy_revenue.jl
src/HSC/write_outputs/write_g2p_capacity.jl
src/HSC/write_outputs/write_h2_capacity.jl
src/HSC/write_outputs/write_h2_storage.jl
src/HSC/write_outputs/write_h2_truck_capacity.jl
Type of change
Please delete options that are not relevant.
Warnings
Warnings are generated. Here is one example:
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration.
Test Configuration:
Checklist: