Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LETKF for single cycle #256

Merged
merged 46 commits into from
Oct 18, 2023
Merged

LETKF for single cycle #256

merged 46 commits into from
Oct 18, 2023

Conversation

CEgerer93
Copy link
Contributor

@CEgerer93 CEgerer93 commented Oct 5, 2023

Description

Completes support of LETKF in geos_atmosphere for a single cycle.
Need to establish conventions for updated ensemble members before cycling in earnest.

Dependencies

No dependencies, though the following needs to be addressed for an error-free Cylc workflow described by suites/letkf/flow.cylc:

  • EvaObservations

Impact

None.

CEgerer93 added 24 commits July 28, 2023 17:07
…etkf/suites_questions.yaml to letkf/suite_questions.yaml
…ents with linked background members at cycle start
@CEgerer93 CEgerer93 requested a review from danholdaway October 5, 2023 21:14
@danholdaway
Copy link
Contributor

Thanks so much for adding this @CEgerer93. You can evaluate the Coding Norm and Code Testing failures on Discover. To do that you would install as follows:

Swell Code Testing:

module use -a /discover/nobackup/drholdaw/opt/modulefiles/core/
module load miniconda/py39_23.3.1
pip install --prefix=/path/to/core/swell/version --no-cache-dir -r requirements-standalone.txt .
module load swell/version
swell_tests_code

Python Coding Norms

cd /path/to/srd/swell/version
python pycodestyle_run.py

The YAML test can't be run on Discover so you have to look at the logs here. It looks like it doesn't like Jinja2 templates in the YAML but what you have is legit. Maybe we should ask @mathomp4 if he knows how to skip a file in the YAML linting.

@CEgerer93
Copy link
Contributor Author

Thank you @danholdaway for the reminder on how to locally check the swell/python norms; minor changes, as expected.

To be clear, swell_tests_code isn't supported in a standard swell build? There are quite a few additional modules (e.g., flake8, etc.) required when installing for the swell code testing.

@danholdaway
Copy link
Contributor

@CEgerer93 that is correct. The problem is that we normally build swell with the spack-stack modules so swell is installed in a manner consistent with how things it will run (i.e. JEDI) are built. However I found that flake8, which the test needs, requires some versions of dependencies newer than what's in spack-stack. So until we can add more to the spack-stack modules we have two installs.

@danholdaway
Copy link
Contributor

Thanks @mathomp4 that suggestion works and is better than what I was doing since it doesn't diverge from the community maintained action for YAML linting.

Copy link
Contributor

@danholdaway danholdaway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for adding this great new feature. Just a few minor things to change please. Also, as I'm reviewing the code I realize that "letkf" might not actually be the correct nomenclature for the suite and tasks etc. Perhaps we should go with LocalEnsembleDa / local_ensemble_da ?

src/swell/deployment/prep_suite.py Outdated Show resolved Hide resolved
src/swell/suites/letkf/flow.cylc Outdated Show resolved Hide resolved
src/swell/tasks/run_jedi_letkf_executable.py Show resolved Hide resolved
src/swell/tasks/task_questions.yaml Show resolved Hide resolved
src/swell/tasks/task_questions.yaml Outdated Show resolved Hide resolved
src/swell/utilities/render_jedi_interface_files.py Outdated Show resolved Hide resolved
Copy link
Contributor Author

@CEgerer93 CEgerer93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed/implemented comments on PR #256.

@CEgerer93 CEgerer93 requested a review from danholdaway October 11, 2023 17:32
@danholdaway
Copy link
Contributor

I sorted out the conflicts and have a few extra comments.

@CEgerer93
Copy link
Contributor Author

Addressed suggestions - perhaps the most important being the relocation of obs localization block into run_jedi_local_ensemble_da_executable.py.

@CEgerer93 CEgerer93 requested a review from danholdaway October 12, 2023 16:53
Copy link
Contributor

@danholdaway danholdaway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this great new feature @CEgerer93.

@danholdaway
Copy link
Contributor

Just running the tier 1 tests to make sure there is no impact elsewhere https://github.com/GEOS-ESM/swell/actions/runs/6562412615

@danholdaway danholdaway merged commit 49f3c56 into develop Oct 18, 2023
@danholdaway danholdaway deleted the feature/letkf-yamls branch October 18, 2023 19:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants