Add workflow for building GEOS and populate relevant tasks #156
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@mathomp4 would appreciate if you have a chance to check over the |
jardizzo
reviewed
Mar 10, 2023
jardizzo
approved these changes
Mar 10, 2023
There was a problem hiding this comment.
Looks like lots of subprocess code consolidation in addition to new tasks. I need to make sure that the nightly CI workflow(s) are not coupled with a now outdated modules file since a new module load was added for GEOS. See one minor comment on possible benefit of "rstrip()" to remove trailing characters (can eliminate the endswith check).
Dooruk
approved these changes
Mar 10, 2023
CRWayman
added a commit
that referenced
this pull request
Mar 31, 2023
* Add the suite file and task shell for running a forecast of GEOS (#150) Co-authored-by: danholdaway <[email protected]> * Remove constraint from the build_jedi suite file (#153) * Add workflow and tasks for performing 3DVar (tested with SOCA) (#152) * Add shell for the new tasks (#133) * add shell for all the new tasaks * rename b clim task * update registry Co-authored-by: danholdaway <[email protected]> * Feature/3dvar generateb (#136) * fetch bump or execution with seg fault * Feature/3dvar standalone godas (#138) * Add the basis for running standalone 3DVar * add bclim to flow * changes in executable * 3dvar is executing now, seg fault is fixed * hofx fix * changes in eva * GODAS observations and golden period * SPECIAL case in executable and modifications for 3dvar EVA * changes in flow * changes in flow and codestyle fixes * trailing spaces Co-authored-by: danholdaway <[email protected]> * write logs in the executable tasks * update YAML files to ioda v3 * Feature/3dvar bump eva (#140) * Add the basis for running standalone 3DVar * add bclim to flow * changes in executable * 3dvar is executing now, seg fault is fixed * hofx fix * changes in eva * GODAS observations and golden period * SPECIAL case in executable and modifications for 3dvar EVA * changes in flow * changes in flow and codestyle fixes * trailing spaces * bump SABER update works * ioda_v3 changes * changes in eva * yaml fixes * yaml fixes --------- Co-authored-by: danholdaway <[email protected]> * refactor eva to allow plotting jedi log things * fixes to workflow files * add new task for generating static b by linking (#147) * add new task for generating static b by linking. * remove prints * correct py norms * yaml norms --------- Co-authored-by: danholdaway <[email protected]> * Feature/3dvar lowres (#151) * fixing workflow * linking changes * lowres works * hofx working * 3dvar workflow running * DA recommended changes --------- Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> * Update discover_nightly.yml (#154) The SWELL Nightly Workflow will now run at 00Z of each day. * Use skylake or cascade lake for the default nodes (#155) Co-authored-by: danholdaway <[email protected]> * Use 3 hours for BuildJedi (#157) Co-authored-by: danholdaway <[email protected]> * Use skylab 3 modules on Discover to maintain compatibility with JEDI (#158) * Use skylab 3 modules on Discover to maintain compatibility with JEDI * skylab3 --------- Co-authored-by: danholdaway <[email protected]> * Add workflow for building GEOS and populate relevant tasks (#156) * add draft tasks for geos clone and build * clone and build working offline * also install * bug * pep8 * use rstrip to remove trailing slash --------- Co-authored-by: danholdaway <[email protected]> * Only use Cascade Lake nodes for the jobs so they can be synced to the nogin node fabric (#159) Co-authored-by: danholdaway <[email protected]> * Fix channels without bc YAML (#160) Co-authored-by: danholdaway <[email protected]> * Bump version of Eva to 1.3.4 (#161) Co-authored-by: danholdaway <[email protected]> * Fix anchors in observation YAML files (#164) Co-authored-by: danholdaway <[email protected]> --------- Co-authored-by: Dan Holdaway <[email protected]> Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> Co-authored-by: jardizzo <[email protected]>
danholdaway
added a commit
that referenced
this pull request
Mar 31, 2023
* Finished refactoring of task list based config prep * Removing questionary from base * Fixing pycodestyle errors * Get YAML Fixes in feature/refactor (#165) * Add the suite file and task shell for running a forecast of GEOS (#150) Co-authored-by: danholdaway <[email protected]> * Remove constraint from the build_jedi suite file (#153) * Add workflow and tasks for performing 3DVar (tested with SOCA) (#152) * Add shell for the new tasks (#133) * add shell for all the new tasaks * rename b clim task * update registry Co-authored-by: danholdaway <[email protected]> * Feature/3dvar generateb (#136) * fetch bump or execution with seg fault * Feature/3dvar standalone godas (#138) * Add the basis for running standalone 3DVar * add bclim to flow * changes in executable * 3dvar is executing now, seg fault is fixed * hofx fix * changes in eva * GODAS observations and golden period * SPECIAL case in executable and modifications for 3dvar EVA * changes in flow * changes in flow and codestyle fixes * trailing spaces Co-authored-by: danholdaway <[email protected]> * write logs in the executable tasks * update YAML files to ioda v3 * Feature/3dvar bump eva (#140) * Add the basis for running standalone 3DVar * add bclim to flow * changes in executable * 3dvar is executing now, seg fault is fixed * hofx fix * changes in eva * GODAS observations and golden period * SPECIAL case in executable and modifications for 3dvar EVA * changes in flow * changes in flow and codestyle fixes * trailing spaces * bump SABER update works * ioda_v3 changes * changes in eva * yaml fixes * yaml fixes --------- Co-authored-by: danholdaway <[email protected]> * refactor eva to allow plotting jedi log things * fixes to workflow files * add new task for generating static b by linking (#147) * add new task for generating static b by linking. * remove prints * correct py norms * yaml norms --------- Co-authored-by: danholdaway <[email protected]> * Feature/3dvar lowres (#151) * fixing workflow * linking changes * lowres works * hofx working * 3dvar workflow running * DA recommended changes --------- Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> * Update discover_nightly.yml (#154) The SWELL Nightly Workflow will now run at 00Z of each day. * Use skylake or cascade lake for the default nodes (#155) Co-authored-by: danholdaway <[email protected]> * Use 3 hours for BuildJedi (#157) Co-authored-by: danholdaway <[email protected]> * Use skylab 3 modules on Discover to maintain compatibility with JEDI (#158) * Use skylab 3 modules on Discover to maintain compatibility with JEDI * skylab3 --------- Co-authored-by: danholdaway <[email protected]> * Add workflow for building GEOS and populate relevant tasks (#156) * add draft tasks for geos clone and build * clone and build working offline * also install * bug * pep8 * use rstrip to remove trailing slash --------- Co-authored-by: danholdaway <[email protected]> * Only use Cascade Lake nodes for the jobs so they can be synced to the nogin node fabric (#159) Co-authored-by: danholdaway <[email protected]> * Fix channels without bc YAML (#160) Co-authored-by: danholdaway <[email protected]> * Bump version of Eva to 1.3.4 (#161) Co-authored-by: danholdaway <[email protected]> * Fix anchors in observation YAML files (#164) Co-authored-by: danholdaway <[email protected]> --------- Co-authored-by: Dan Holdaway <[email protected]> Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> Co-authored-by: jardizzo <[email protected]> * Removed white space for code styling --------- Co-authored-by: Dan Holdaway <[email protected]> Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> Co-authored-by: jardizzo <[email protected]>
CRWayman
added a commit
that referenced
this pull request
May 4, 2023
* Finished refactoring of task list based config prep * Removing questionary from base * Fixing pycodestyle errors * Get YAML Fixes in feature/refactor (#165) * Add the suite file and task shell for running a forecast of GEOS (#150) Co-authored-by: danholdaway <[email protected]> * Remove constraint from the build_jedi suite file (#153) * Add workflow and tasks for performing 3DVar (tested with SOCA) (#152) * Add shell for the new tasks (#133) * add shell for all the new tasaks * rename b clim task * update registry Co-authored-by: danholdaway <[email protected]> * Feature/3dvar generateb (#136) * fetch bump or execution with seg fault * Feature/3dvar standalone godas (#138) * Add the basis for running standalone 3DVar * add bclim to flow * changes in executable * 3dvar is executing now, seg fault is fixed * hofx fix * changes in eva * GODAS observations and golden period * SPECIAL case in executable and modifications for 3dvar EVA * changes in flow * changes in flow and codestyle fixes * trailing spaces Co-authored-by: danholdaway <[email protected]> * write logs in the executable tasks * update YAML files to ioda v3 * Feature/3dvar bump eva (#140) * Add the basis for running standalone 3DVar * add bclim to flow * changes in executable * 3dvar is executing now, seg fault is fixed * hofx fix * changes in eva * GODAS observations and golden period * SPECIAL case in executable and modifications for 3dvar EVA * changes in flow * changes in flow and codestyle fixes * trailing spaces * bump SABER update works * ioda_v3 changes * changes in eva * yaml fixes * yaml fixes --------- Co-authored-by: danholdaway <[email protected]> * refactor eva to allow plotting jedi log things * fixes to workflow files * add new task for generating static b by linking (#147) * add new task for generating static b by linking. * remove prints * correct py norms * yaml norms --------- Co-authored-by: danholdaway <[email protected]> * Feature/3dvar lowres (#151) * fixing workflow * linking changes * lowres works * hofx working * 3dvar workflow running * DA recommended changes --------- Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> * Update discover_nightly.yml (#154) The SWELL Nightly Workflow will now run at 00Z of each day. * Use skylake or cascade lake for the default nodes (#155) Co-authored-by: danholdaway <[email protected]> * Use 3 hours for BuildJedi (#157) Co-authored-by: danholdaway <[email protected]> * Use skylab 3 modules on Discover to maintain compatibility with JEDI (#158) * Use skylab 3 modules on Discover to maintain compatibility with JEDI * skylab3 --------- Co-authored-by: danholdaway <[email protected]> * Add workflow for building GEOS and populate relevant tasks (#156) * add draft tasks for geos clone and build * clone and build working offline * also install * bug * pep8 * use rstrip to remove trailing slash --------- Co-authored-by: danholdaway <[email protected]> * Only use Cascade Lake nodes for the jobs so they can be synced to the nogin node fabric (#159) Co-authored-by: danholdaway <[email protected]> * Fix channels without bc YAML (#160) Co-authored-by: danholdaway <[email protected]> * Bump version of Eva to 1.3.4 (#161) Co-authored-by: danholdaway <[email protected]> * Fix anchors in observation YAML files (#164) Co-authored-by: danholdaway <[email protected]> --------- Co-authored-by: Dan Holdaway <[email protected]> Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> Co-authored-by: jardizzo <[email protected]> * Removed white space for code styling * Fixed change choices bug * Removed erroneous print statement * Removed unecessary before_next function which should be in cli class * Removing commented suite option for click --------- Co-authored-by: Dan Holdaway <[email protected]> Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> Co-authored-by: jardizzo <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add tasks for clone, build (by linking) and build GEOS.
Add workflow for building GEOS. First part of having a workflow for making a forecast with GEOS.