Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Read derived variables from other MIP tables #2256

Merged
merged 3 commits into from
Jan 18, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions doc/recipe/preprocessor.rst
Original file line number Diff line number Diff line change
Expand Up @@ -201,13 +201,15 @@ the ozone 3D field. In this case, a derivation function is provided to
vertically integrate the ozone and obtain total column ozone for direct
comparison with the observations.

To contribute a new derived variable, it is also necessary to define a name for
it and to provide the corresponding CMOR table. This is to guarantee the proper
metadata definition is attached to the derived data. Such custom CMOR tables
are collected as part of the `ESMValCore package
<https://github.com/ESMValGroup/ESMValCore>`_. By default, the variable
derivation will be applied only if the variable is not already available in the
input data, but the derivation can be forced by setting the appropriate flag.
The tool will also look in other ``mip`` tables for the same ``project`` to find
the definition of derived variables. To contribute a completely new derived
variable, it is necessary to define a name for it and to provide the
corresponding CMOR table. This is to guarantee the proper metadata definition
is attached to the derived data. Such custom CMOR tables are collected as part
of the `ESMValCore package <https://github.com/ESMValGroup/ESMValCore/tree/main/esmvalcore/cmor/tables/custom>`_.
By default, the variable derivation will be applied only if the variable is not
already available in the input data, but the derivation can be forced by
setting the ``force_derivation`` flag.

.. code-block:: yaml

Expand Down
8 changes: 4 additions & 4 deletions esmvalcore/cmor/table.py
Original file line number Diff line number Diff line change
Expand Up @@ -296,8 +296,8 @@ def get_variable(
pass

# If that didn't work, look in all tables (i.e., other MIPs) if
# cmor_strict=False
var_info = self._look_in_all_tables(alt_names_list)
# cmor_strict=False or derived=True
var_info = self._look_in_all_tables(derived, alt_names_list)

# If that didn' work either, look in default table if cmor_strict=False
# or derived=True
Expand All @@ -324,10 +324,10 @@ def _look_in_default(self, derived, alt_names_list, table_name):
break
return var_info

def _look_in_all_tables(self, alt_names_list):
def _look_in_all_tables(self, derived, alt_names_list):
"""Look for variable in all tables."""
var_info = None
if not self.strict:
if (not self.strict or derived):
for alt_names in alt_names_list:
var_info = self._look_all_tables(alt_names)
if var_info:
Expand Down
5 changes: 5 additions & 0 deletions tests/integration/cmor/test_table.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,11 @@ def test_get_variable_from_alt_names(self):
var = self.variables_info.get_variable('SImon', 'sic')
self.assertEqual(var.short_name, 'siconc')

def test_get_variable_derived(self):
"""Test that derived variable are looked up from other MIP tables."""
var = self.variables_info.get_variable('3hr', 'sfcWind', derived=True)
self.assertEqual(var.short_name, 'sfcWind')

def test_get_variable_from_custom(self):
"""Get a variable from default."""
self.variables_info.strict = False
Expand Down