Feature/jjin3/ufo tests x0048 assign error #204
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* x48 satwind yaml * clean up and benchmark change * removing @s * yaml fix
* Update configurations for microwave data in x0048. * Code norm. * Code norm. * Code norm. * Code norm. * Code norm.
Co-authored-by: Dan Holdaway <[email protected]>
gmao-jjin3
approved these changes
Aug 11, 2023
danholdaway
added a commit
that referenced
this pull request
Aug 14, 2023
* fix a few questions * fixes for r2d2 * extensive override * bug fixes when adding string * remove extra model components key * clean up experiment yaml * Feature/jjin3/ufo tests x0048 assign error (#204) * add suite for obs filter tests * add ncdiag tasks * converter working (no geovals) * executable runs * sat bias correction conversion added * add aircraft yaml * move to R2D2 for getting the observations * convert to using GetObservations * append * uncomment * clean up hidden aspects of the config * move the opening of jedi interface files * add class to handle jedi file templating * stage tasks working * unify copyright date * pep8 * add utility for setting the list of tasks for each question * add question prompts * add more questions and defaults * move jedi exe base * refactor run jedi exe base * remove fgat as not needed * move depends * write lists the way python does * make sure jedi_forecast model is used * more q clean up * remove jedi exe base class from questions * convert task names to CamelCase in the questions * modify config class to create variables and methods to get them * Update generate_b_climatology_by_linking.py quick fix * common config availability * eva obs fix * use new variable functions for config instead of config_get * fix clean patterns * update question yaml generation for now not using config_get * change jedi build/source in platform * add code for genrating default dictionaries * fix the setup py paths * move task questions * initialize the task questions for components * init def dict atmos * use existing for platform * set the defaults * standardize the yamls * add geos directories * add new suites * name change in config * fixes to get hofx and 3dvar running * remove num iterations from RunJediHofx task * coding norms * add dictionary gneeration test * add test_suite action * action for test suite * add init to test path * copy task qustions to install dir * action job name * build jedi paths * make tasks complianct with new config and base delgations * Add questions to task_questions * Add questions to task_questions formatting * add question answers to model interface * Do not overwrite the dictionary in the source code * more checking * improve testing of dictionary creation * Use spack-stack 1.4 modules to maintain compatibility with JEDI * check for defer_to_ in the files * reduce prints in tests * py coding norms * unload crtm and update jedi_bundle * unused variables * rename suite * Bug fix and remove print * add operator test capability * rename some things * temp comment out * YAML generation working * add geos adas suite * Tasks for geosadas running * dates * unused variables * standardise task_questions * update default dictionaries * config changes * GMI_GPM full configuration for IODA v3 and x0048. (#187) * GMI_GPM full configuration for IODA v3 and x0048. * Code norm. * Code norm. * update data paths * all tasks running smoothly * rename workflows * coding norms * yaml norms * test suite * replace tier2 * tier2 name * tier 2 take 2 * remove trigger for tier1 * update target workflows * update target workflows branch * some renaming * action testing * action testing * bug fix in run_command print * bug fix in run_command print * action testing * action testing * flow bug * add alternative workflow that generates the geovals as well as obs * better requirements * add da run parameter * ufo_testing runs in cylc * update hofx to be x0048 * updating scatwind yaml (#194) * UFO Tests - Satwind (#193) * x48 satwind yaml * clean up and benchmark change * removing @s * yaml fix * add some missing instruments * add gps and avhrr specific cases * Feature/jjin3/ufo tests x0048 micorwave (#197) * Update configurations for microwave data in x0048. * Code norm. * Code norm. * Code norm. * Code norm. * Code norm. * YAML files for CrIS-FSR(n20,npp), AIRS & SSMIS (#200) Co-authored-by: Dan Holdaway <[email protected]> * clean up issues in ncdiag 2 ioda calling task * fix bug for single file outputs * updating YAML resource files for IASI (#202) * ozone working better * do not always create the forecast directory * Change the name of the filter assign observational error to "Perform Action". * revert to develop --------- Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> Co-authored-by: dooruk <[email protected]> Co-authored-by: Jianjun Jin <[email protected]> Co-authored-by: Akira Sewnath <[email protected]> Co-authored-by: Wei Gu <[email protected]> Co-authored-by: gmao-jjin3 <[email protected]> * put suite files where they should be * can create experiment for all suites * override bug fixes * reorganize testing directory * generates the same 3dvar_cycle yaml as before * remove print * copy suite test yamls to install * clean up ocean yaml * work on 3dvar * same experiment yamls as develop * same experiment yamls as develop hofx * fix eva issue * Use existing build for var runs * allow users to override some slurm variables * allow for both test and use override * Remove some rogue system files * coding norms --------- Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> Co-authored-by: dooruk <[email protected]> Co-authored-by: Jianjun Jin <[email protected]> Co-authored-by: Akira Sewnath <[email protected]> Co-authored-by: Wei Gu <[email protected]> Co-authored-by: gmao-jjin3 <[email protected]>
danholdaway
added a commit
that referenced
this pull request
Aug 14, 2023
* add suite for obs filter tests * add ncdiag tasks * converter working (no geovals) * executable runs * sat bias correction conversion added * add aircraft yaml * move to R2D2 for getting the observations * convert to using GetObservations * append * uncomment * clean up hidden aspects of the config * move the opening of jedi interface files * add class to handle jedi file templating * stage tasks working * unify copyright date * pep8 * add utility for setting the list of tasks for each question * add question prompts * add more questions and defaults * move jedi exe base * refactor run jedi exe base * remove fgat as not needed * move depends * write lists the way python does * make sure jedi_forecast model is used * more q clean up * remove jedi exe base class from questions * convert task names to CamelCase in the questions * modify config class to create variables and methods to get them * Update generate_b_climatology_by_linking.py quick fix * common config availability * eva obs fix * use new variable functions for config instead of config_get * fix clean patterns * update question yaml generation for now not using config_get * change jedi build/source in platform * add code for genrating default dictionaries * fix the setup py paths * move task questions * initialize the task questions for components * init def dict atmos * use existing for platform * set the defaults * standardize the yamls * add geos directories * add new suites * name change in config * swell prepare config from feature/refactor * commit prep config python code for questionary from feature/refactor * Debugging build_jedi with new framework * fixes to get hofx and 3dvar running * remove num iterations from RunJediHofx task * coding norms * add dictionary gneeration test * add test_suite action * action for test suite * add init to test path * copy task qustions to install dir * action job name * build jedi paths * make tasks complianct with new config and base delgations * Add questions to task_questions * Add questions to task_questions formatting * add question answers to model interface * Do not overwrite the dictionary in the source code * more checking * improve testing of dictionary creation * Use spack-stack 1.4 modules to maintain compatibility with JEDI * check for defer_to_ in the files * reduce prints in tests * py coding norms * unload crtm and update jedi_bundle * Removing unnecessary suite_questions yaml from suites_new * Adding build jedi suite directory * Removing models as a cli option * Removing experiment id default since we are not using jinja * Updating prep config tool for use with new task_questions yaml * changing one type from string to string-drop-list * unused variables * rename suite * Bug fix and remove print * add operator test capability * rename some things * temp comment out * YAML generation working * add geos adas suite * Tasks for geosadas running * dates * unused variables * standardise task_questions * update default dictionaries * config changes * Changing the input method flad back to m * Refactoring of task questions yaml introduced to prep config python codes * Adding 'ask_question' keys to yaml * Changed duration to iso-duration in some question types * Fixing pycodestyle issues * remove files left by merge * exchange the suites directories * suites_new -> suites * repair some merge conflict errors * add basis for model based suite questions * GMI_GPM full configuration for IODA v3 and x0048. (#187) * GMI_GPM full configuration for IODA v3 and x0048. * Code norm. * Code norm. * update data paths * all tasks running smoothly * rename workflows * coding norms * yaml norms * test suite * replace tier2 * tier2 name * tier 2 take 2 * remove trigger for tier1 * update target workflows * update target workflows branch * some renaming * action testing * action testing * bug fix in run_command print * bug fix in run_command print * action testing * action testing * flow bug * Moved the src dir * Moved src directory * Adding catch for ctrl+c * Moved into newconf dir * add alternative workflow that generates the geovals as well as obs * better requirements * add da run parameter * Refactoring to add model suite questions * prep_config_base.py * ufo_testing runs in cylc * Fixing directory order * update hofx to be x0048 * updating scatwind yaml (#194) * UFO Tests - Satwind (#193) * x48 satwind yaml * clean up and benchmark change * removing @s * yaml fix * add some missing instruments * add gps and avhrr specific cases * Feature/jjin3/ufo tests x0048 micorwave (#197) * Update configurations for microwave data in x0048. * Code norm. * Code norm. * Code norm. * Code norm. * Code norm. * YAML files for CrIS-FSR(n20,npp), AIRS & SSMIS (#200) Co-authored-by: Dan Holdaway <[email protected]> * clean up issues in ncdiag 2 ioda calling task * fix bug for single file outputs * Removing debug prints and adding comments to platform and suite_to_run output * updating YAML resource files for IASI (#202) * ozone working better * do not always create the forecast directory * more updates towards trifecta * residual merge issue * type error after merge * Feature/questions to tasks dan (#205) * fix a few questions * fixes for r2d2 * extensive override * bug fixes when adding string * remove extra model components key * clean up experiment yaml * Feature/jjin3/ufo tests x0048 assign error (#204) * add suite for obs filter tests * add ncdiag tasks * converter working (no geovals) * executable runs * sat bias correction conversion added * add aircraft yaml * move to R2D2 for getting the observations * convert to using GetObservations * append * uncomment * clean up hidden aspects of the config * move the opening of jedi interface files * add class to handle jedi file templating * stage tasks working * unify copyright date * pep8 * add utility for setting the list of tasks for each question * add question prompts * add more questions and defaults * move jedi exe base * refactor run jedi exe base * remove fgat as not needed * move depends * write lists the way python does * make sure jedi_forecast model is used * more q clean up * remove jedi exe base class from questions * convert task names to CamelCase in the questions * modify config class to create variables and methods to get them * Update generate_b_climatology_by_linking.py quick fix * common config availability * eva obs fix * use new variable functions for config instead of config_get * fix clean patterns * update question yaml generation for now not using config_get * change jedi build/source in platform * add code for genrating default dictionaries * fix the setup py paths * move task questions * initialize the task questions for components * init def dict atmos * use existing for platform * set the defaults * standardize the yamls * add geos directories * add new suites * name change in config * fixes to get hofx and 3dvar running * remove num iterations from RunJediHofx task * coding norms * add dictionary gneeration test * add test_suite action * action for test suite * add init to test path * copy task qustions to install dir * action job name * build jedi paths * make tasks complianct with new config and base delgations * Add questions to task_questions * Add questions to task_questions formatting * add question answers to model interface * Do not overwrite the dictionary in the source code * more checking * improve testing of dictionary creation * Use spack-stack 1.4 modules to maintain compatibility with JEDI * check for defer_to_ in the files * reduce prints in tests * py coding norms * unload crtm and update jedi_bundle * unused variables * rename suite * Bug fix and remove print * add operator test capability * rename some things * temp comment out * YAML generation working * add geos adas suite * Tasks for geosadas running * dates * unused variables * standardise task_questions * update default dictionaries * config changes * GMI_GPM full configuration for IODA v3 and x0048. (#187) * GMI_GPM full configuration for IODA v3 and x0048. * Code norm. * Code norm. * update data paths * all tasks running smoothly * rename workflows * coding norms * yaml norms * test suite * replace tier2 * tier2 name * tier 2 take 2 * remove trigger for tier1 * update target workflows * update target workflows branch * some renaming * action testing * action testing * bug fix in run_command print * bug fix in run_command print * action testing * action testing * flow bug * add alternative workflow that generates the geovals as well as obs * better requirements * add da run parameter * ufo_testing runs in cylc * update hofx to be x0048 * updating scatwind yaml (#194) * UFO Tests - Satwind (#193) * x48 satwind yaml * clean up and benchmark change * removing @s * yaml fix * add some missing instruments * add gps and avhrr specific cases * Feature/jjin3/ufo tests x0048 micorwave (#197) * Update configurations for microwave data in x0048. * Code norm. * Code norm. * Code norm. * Code norm. * Code norm. * YAML files for CrIS-FSR(n20,npp), AIRS & SSMIS (#200) Co-authored-by: Dan Holdaway <[email protected]> * clean up issues in ncdiag 2 ioda calling task * fix bug for single file outputs * updating YAML resource files for IASI (#202) * ozone working better * do not always create the forecast directory * Change the name of the filter assign observational error to "Perform Action". * revert to develop --------- Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> Co-authored-by: dooruk <[email protected]> Co-authored-by: Jianjun Jin <[email protected]> Co-authored-by: Akira Sewnath <[email protected]> Co-authored-by: Wei Gu <[email protected]> Co-authored-by: gmao-jjin3 <[email protected]> * put suite files where they should be * can create experiment for all suites * override bug fixes * reorganize testing directory * generates the same 3dvar_cycle yaml as before * remove print * copy suite test yamls to install * clean up ocean yaml * work on 3dvar * same experiment yamls as develop * same experiment yamls as develop hofx * fix eva issue * Use existing build for var runs * allow users to override some slurm variables * allow for both test and use override * Remove some rogue system files * coding norms --------- Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> Co-authored-by: dooruk <[email protected]> Co-authored-by: Jianjun Jin <[email protected]> Co-authored-by: Akira Sewnath <[email protected]> Co-authored-by: Wei Gu <[email protected]> Co-authored-by: gmao-jjin3 <[email protected]> * cnorm * code test issue --------- Co-authored-by: danholdaway <[email protected]> Co-authored-by: Doruk Ardağ <[email protected]> Co-authored-by: dooruk <[email protected]> Co-authored-by: Dan Holdaway <[email protected]> Co-authored-by: Jianjun Jin <[email protected]> Co-authored-by: Akira Sewnath <[email protected]> Co-authored-by: Wei Gu <[email protected]> Co-authored-by: gmao-jjin3 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Reopen @gmao-jjin3 changes