Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest.approx considers boolean numeric types #9353

Closed
The-Compiler opened this issue Nov 30, 2021 · 1 comment · Fixed by #9354
Closed

pytest.approx considers boolean numeric types #9353

The-Compiler opened this issue Nov 30, 2021 · 1 comment · Fixed by #9354
Labels
topic: approx related to pytest.approx function

Comments

@The-Compiler
Copy link
Member

The-Compiler commented Nov 30, 2021

From the docs:

You can also use approx to compare nonnumeric types, or dicts and sequences containing nonnumeric types, in which case it falls back to strict equality. This can be useful for comparing dicts and sequences that can contain optional values:

>>> {"required": 1.0000005, "optional": None} == approx({"required": 1, "optional": None})
True
>>> [None, 1.0000005] == approx([None,1])
True
>>> ["foo", 1.0000005] == approx([None,1])
False

However, that seems to treat booleans as numerical values too:

import pytest

def test_approx():
    d = {"value": 0.3, "active": True}
    expected = {"value": 0.1 + 0.2, "active": False}
    assert d == pytest.approx(expected)

results in:

AssertionError: assert {'active': True, 'value': 0.3} == approx({'value': 0.30000000000000004 ± 3.0e-07, 'active': False ± 1.0e-12})

(on Python 3.9.7 and pytest 6.2.5). Note the False ± 1.0e-12. While this is technically correct, it doesn't seem like a very reasonable way of comparing booleans.

cc @jvansanten who seems to have implemented this originally in #7710.

@The-Compiler The-Compiler added the topic: approx related to pytest.approx function label Nov 30, 2021
@jvansanten
Copy link
Contributor

I'm not so sure what to do about this. The idea of #7710 was to use approximate comparison whenever (expected-obtained) was allowed by the target types, only falling back to equality for non arithmetic types. int-bool is allowed (in contrast to Enum-Enum), so True == approx(False, abs=2) is technically correct as you point out.

One straightforward solution to special-case the equality fallback, so approximate comparisons are used for all subclasses of Decimal and Complex, but not for bool.

jvansanten added a commit to jvansanten/pytest that referenced this issue Nov 30, 2021
patchback bot pushed a commit that referenced this issue Nov 29, 2024
Fixes #9353

(cherry picked from commit a16e8ea)
nicoddemus pushed a commit that referenced this issue Dec 1, 2024
Fixes #9353

(cherry picked from commit a16e8ea)

Co-authored-by: Jakob van Santen <[email protected]>
DougBurke added a commit to DougBurke/sherpa that referenced this issue Dec 6, 2024
Fix sherpa#2202

In pytest 8.3.3 and earlier

    >>> import numpy as np; import pytest
    >>> pytest.__version__
    '8.3.3'
    >>> np.ones(2, dtype=bool) == pytest.approx([True, True])
    True

However, pytest 8.3.4 now causes this to fail

    >>> import numpy as np; import pytest
    >>> pytest.__version__
    '8.3.4'
    >>> np.ones(2, dtype=bool) == pytest.approx([True, True])
    False

This is because of "pytest.approx considers boolean numeric types"
pytest-dev/pytest#9353

The solution is to make the "expected" value be a ndarray, and so

    >>> np.ones(2, dtype=bool) == pytest.approx(np.asarray([True, True]))
    True

holds with both pytest 8.3.3 and 8.3.4.

So this commit basically goes through and updates the tests so that
we use a ndarray for boolean arrays.

An alternative would be to change from

    assert got == pytest.approx(expected)

to something like

    assert np.all(got == expected)

However, the error message when the array lengths are different or
an element is different are a **lot less** useful, and the change would
be even-more invasive than this change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: approx related to pytest.approx function
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants