Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dynamic_context is not set for tests which are static or class methods #1923

Open
james-garner-canonical opened this issue Feb 5, 2025 · 2 comments
Labels
bug Something isn't working

Comments

@james-garner-canonical
Copy link

Describe the bug
When using dynamic_context=test_function, the context is not set if the test function is a @staticmethod or @classmethod.

I believe this is due to the logic of context.qualname_from_frame, which explicitly tests for the presence of a first argument named "self" to detect whether a test function is scoped to a class. Since static and class methods do not tend to have such an argument, they are treated as functions, but because they are not found in f_globals, None is returned by qualname_from_frame and in turn returned by should_start_context_test_function, as if the method was not a test at all.

To Reproduce

Here is a minimal example that demonstrates the problem.

  1. place the files below alongside each other in some directory
  2. run tox in that directory
  3. note that this warning is emitted CoverageWarning: No contexts were measured self.coverage._warn("No contexts were measured")
  4. observe that the context for the lines of foo_module.py are labeled (empty) in the htmlcov report

foo_module.py

def foo_fn():
    print('foo')

test_foo.py

import foo_module

class TestFoo:
    @staticmethod
    def test_foo_static():
        foo_module.foo_fn()

    @classmethod
    def test_foo_class(cls):
        foo_module.foo_fn()

.coveragerc

[run]
dynamic_context = test_function

tox.ini

[testenv]
deps =
    coverage
    pytest
commands =
    coverage run -m pytest
    coverage html --show-contexts

Expected behavior
I would expect to see the contexts TestFoo.test_foo_static and TestFoo.test_foo_class in the htmlcov report, and no warning.

Additional context
Class methods can be covered by adding an explicit test for a first argument named "cls" similar to the existing logic.

Static methods are a little trickier. Python 3.11 adds the co_qualname attribute to frame.f_code, which is exactly what's needed, and solves the problem for me. For older Python versions, the status quo could be improved by simply returning something like staticmethod(test_foo_static).

For example, changing these lines as follows might work. The only caveat is that perhaps there's some other reason you could reach the func is None case.

    if method is None:
        func = frame.f_globals.get(fname)
        if func is None:
            try:
                return frame.f_code.co_qualname
            except AttributeError:
                return f"staticmethod({fname})"
        return cast(str, func.__module__ + "." + fname)

  1. What version of Python are you using? I've reproduced this on versions 3.8 - 3.13
  2. What version of coverage.py shows the problem? Coverage.py, version 7.6.10 with C extension. I believe this issue is present in main.
  3. What versions of what packages do you have installed? See minimal example.
  4. What code shows the problem? See minimal example.
  5. What commands should we run to reproduce the problem? See minimal example.
@james-garner-canonical james-garner-canonical added the bug Something isn't working label Feb 5, 2025
@nedbat
Copy link
Owner

nedbat commented Feb 5, 2025

@james-garner-canonical I guess I've never seen @staticmethod or @classmethod test methods before. Where do you use them?

While I have you, would you be able to influence Canonical to provide some GitHub sponsorship for coverage.py?

@james-garner-canonical
Copy link
Author

@james-garner-canonical I guess I've never seen @staticmethod or @classmethod test methods before. Where do you use them?

Hi @nedbat , I mostly just use @staticmethod because it seemed nicer to me than having an unused self argument. It's probably a bit idiosyncratic, and I do know that support for @staticmethod was added to pytest quite late (2017), but I was still surprised when I eventually realised why coverage wasn't setting the dynamic context.

While I have you, would you be able to influence Canonical to provide some GitHub sponsorship for coverage.py?

Good idea, I'm not in charge of such things, but I'll ask around

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants