Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Systematize calculation testing #1214

Open
dopplershift opened this issue Oct 21, 2019 · 3 comments
Open

Systematize calculation testing #1214

dopplershift opened this issue Oct 21, 2019 · 3 comments
Labels
Area: Tests Affects tests Type: Feature New functionality

Comments

@dopplershift
Copy link
Member

dopplershift commented Oct 21, 2019

It would probably be good to find some way to make our calculation testing more systematic, in terms of testing certain features:

This was inspired by #1209 and other issues. I'm not sure how to go about it. PyTest parameterization and/or fixtures could be useful, but I'm not sure how to handle encoding truth values with those. I suppose those could be smoke tests for those input cases, though checking behavior with NaNs and masked arrays could need some truth values for correctness.

@dopplershift dopplershift added Type: Feature New functionality Area: Tests Affects tests labels Oct 21, 2019
@jthielen
Copy link
Collaborator

With regards to mutating inputs in-place, I've opened hgrecco/pint#925 as an upstream issue.

@rpmanser
Copy link
Contributor

rpmanser commented Jun 2, 2020

Fixtures could be made for each data type with meteorological variables within them and parameterizing could take care of edge cases.

For example, in the conftest.py file there would be:

@pytest.fixture
def arrays():
    return {
        'u': np.array([4., 2., 0., 0.]) * units('m/s'),
        'v': np.array([0., 2., 4., 0.]) * units('m/s'),
        # More variables here...
    }

Then a test would accept one of the data type fixtures and select the variables that are relevant to the function being tested:

def test_direction_array(arrays):
    """Test calculating wind direction."""
    direc = wind_direction(arrays['u'], arrays['v'])
    true_dir = np.array([270., 225., 180., 0.]) * units.deg

    assert_array_almost_equal(true_dir, direc, 4)

Something similar could be done for scalars, masked arrays, and NaNs.

Then for edge cases, parametrize could be used.

@pytest.mark.parametrize(
    "u, v, true_dir",
    [
        # Edge case for northerly and calm winds
        pytest.param(
            np.array([0., -0., 0.]) * units('m/s'),
            np.array([0., 0., -5.]) * units('m/s'),
            np.array([0., 0., 360.]) * units.deg
        )
        # Next edge case here
    ]
)
def test_direction(u, v, true_dir):
    """Test edge cases for wind direction"""
    direc = wind_direction(u, v)

    assert_array_almost_equal(true_dir, direc, 4)

I'm not sure how to handle truth values except for defining them in the test function, which is how everything is set up currently. There are also many tests where exceptions are raised, or specific units are being checked, which would probably remain untouched.

Anyway, would the fixtures be a good way to go about systematically testing data types in each function? And are there thoughts on parameterizing edge cases?

rpmanser added a commit to rpmanser/MetPy that referenced this issue Jun 3, 2020
This commit is a draft of implementing unit tests for data types in each
calc function and parameterizing tests for edge cases.

- Add fixtures in conftest.py containing test data for each supported
data type.
- Add/update tests for individual data types using fixtures.
- Group tests into a class.
- Add fixture for truth values specific to the test class.
- Parameterize the edge case.

Makes progress toward closing Unidata#1214
rpmanser added a commit to rpmanser/MetPy that referenced this issue Jun 3, 2020
This commit is a draft of implementing unit tests for data types in each
calc function and parameterizing tests for edge cases.

- Add fixtures in conftest.py containing test data for each supported
data type.
- Add/update tests for individual data types using fixtures.
- Group tests into a class.
- Add fixture for truth values specific to the test class.
- Parameterize the edge case.

Makes progress toward closing Unidata#1214
rpmanser added a commit to rpmanser/MetPy that referenced this issue Sep 15, 2020
This commit is a draft of implementing unit tests for data types in each
calc function and parameterizing tests for edge cases.

- Add fixtures in conftest.py containing test data for each supported
data type.
- Add/update tests for individual data types using fixtures.
- Group tests into a class.
- Add fixture for truth values specific to the test class.
- Parameterize the edge case.

Makes progress toward closing Unidata#1214
@dopplershift
Copy link
Member Author

Adding another item to the list: For functions that deal with temperature (at least anywhere we manually "handle" units), verify that calculations work with degC/degF.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Tests Affects tests Type: Feature New functionality
Projects
None yet
Development

No branches or pull requests

3 participants