Feature/questions to tasks callum #189
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Update configurations for microwave data in x0048. * Code norm. * Code norm. * Code norm. * Code norm. * Code norm.
Co-authored-by: Dan Holdaway <[email protected]>
|
Hey Dan, I noticed an issue with the build_jedi task questions: "Do you want to use an existing JEDI build or create a new build?" should probably be a string-drop-list so users don't have to guess the syntax for "create" instead of use_existing |
…m/GEOS-ESM/swell into feature/questions_to_tasks-callum
|
Ok I've fixed the experiment config output for suite_to_run and platform by adding them to my dictionary in the "suite_setup" function in prep_config_base.py I also removed unnecessary print statements, and removed the "Please answer the following questions..." text when defaults is called, this question is still printed when questionary is used. |
Oh yeah. That makes sense. It should be a drop list. |
|
Thanks for the latest updates. |
* fix a few questions * fixes for r2d2 * extensive override * bug fixes when adding string * remove extra model components key * clean up experiment yaml * Feature/jjin3/ufo tests x0048 assign error (#204) * add suite for obs filter tests * add ncdiag tasks * converter working (no geovals) * executable runs * sat bias correction conversion added * add aircraft yaml * move to R2D2 for getting the observations * convert to using GetObservations * append * uncomment * clean up hidden aspects of the config * move the opening of jedi interface files * add class to handle jedi file templating * stage tasks working * unify copyright date * pep8 * add utility for setting the list of tasks for each question * add question prompts * add more questions and defaults * move jedi exe base * refactor run jedi exe base * remove fgat as not needed * move depends * write lists the way python does * make sure jedi_forecast model is used * more q clean up * remove jedi exe base class from questions * convert task names to CamelCase in the questions * modify config class to create variables and methods to get them * Update generate_b_climatology_by_linking.py quick fix * common config availability * eva obs fix * use new variable functions for config instead of config_get * fix clean patterns * update question yaml generation for now not using config_get * change jedi build/source in platform * add code for genrating default dictionaries * fix the setup py paths * move task questions * initialize the task questions for components * init def dict atmos * use existing for platform * set the defaults * standardize the yamls * add geos directories * add new suites * name change in config * fixes to get hofx and 3dvar running * remove num iterations from RunJediHofx task * coding norms * add dictionary gneeration test * add test_suite action * action for test suite * add init to test path * copy task qustions to install dir * action job name * build jedi paths * make tasks complianct with new config and base delgations * Add questions to task_questions * Add questions to task_questions formatting * add question answers to model interface * Do not overwrite the dictionary in the source code * more checking * improve testing of dictionary creation * Use spack-stack 1.4 modules to maintain compatibility with JEDI * check for defer_to_ in the files * reduce prints in tests * py coding norms * unload crtm and update jedi_bundle * unused variables * rename suite * Bug fix and remove print * add operator test capability * rename some things * temp comment out * YAML generation working * add geos adas suite * Tasks for geosadas running * dates * unused variables * standardise task_questions * update default dictionaries * config changes * GMI_GPM full configuration for IODA v3 and x0048. (#187) * GMI_GPM full configuration for IODA v3 and x0048. * Code norm. * Code norm. * update data paths * all tasks running smoothly * rename workflows * coding norms * yaml norms * test suite * replace tier2 * tier2 name * tier 2 take 2 * remove trigger for tier1 * update target workflows * update target workflows branch * some renaming * action testing * action testing * bug fix in run_command print * bug fix in run_command print * action testing * action testing * flow bug * add alternative workflow that generates the geovals as well as obs * better requirements * add da run parameter * ufo_testing runs in cylc * update hofx to be x0048 * updating scatwind yaml (#194) * UFO Tests - Satwind (#193) * x48 satwind yaml * clean up and benchmark change * removing @s * yaml fix * add some missing instruments * add gps and avhrr specific cases * Feature/jjin3/ufo tests x0048 micorwave (#197) * Update configurations for microwave data in x0048. * Code norm. * Code norm. * Code norm. * Code norm. * Code norm. * YAML files for CrIS-FSR(n20,npp), AIRS & SSMIS (#200) Co-authored-by: Dan Holdaway <[email protected]> * clean up issues in ncdiag 2 ioda calling task * fix bug for single file outputs * updating YAML resource files for IASI (#202) * ozone working better * do not always create the forecast directory * Change the name of the filter assign observational error to "Perform Action". * revert to develop --------- Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> Co-authored-by: dooruk <[email protected]> Co-authored-by: Jianjun Jin <[email protected]> Co-authored-by: Akira Sewnath <[email protected]> Co-authored-by: Wei Gu <[email protected]> Co-authored-by: gmao-jjin3 <[email protected]> * put suite files where they should be * can create experiment for all suites * override bug fixes * reorganize testing directory * generates the same 3dvar_cycle yaml as before * remove print * copy suite test yamls to install * clean up ocean yaml * work on 3dvar * same experiment yamls as develop * same experiment yamls as develop hofx * fix eva issue * Use existing build for var runs * allow users to override some slurm variables * allow for both test and use override * Remove some rogue system files * coding norms --------- Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> Co-authored-by: dooruk <[email protected]> Co-authored-by: Jianjun Jin <[email protected]> Co-authored-by: Akira Sewnath <[email protected]> Co-authored-by: Wei Gu <[email protected]> Co-authored-by: gmao-jjin3 <[email protected]>
danholdaway
approved these changes
Aug 14, 2023
|
Thanks for this substantial piece of work @CRWayman. The changes are now merged into develop. Cheers to the removal of a significant amount of duplicate YAML and a clearer experiment generation system for everyone. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Large update to prep config python codes in line with new task questions yaml structure.
Dependencies
Impact