Inconsistent behaviour of recwarn

[Themanwithoutaplan]

I'm have a test that checks that a warning is issued. This runs perfectly with pytest but can fail when using tox. I suspect that it has something to do with the warningsregistry but I can't see offhand.

A traceback can be seen at: https://build.adimian.com/job/openpyxl-2.3/70/consoleFull

The failure happens when testing is not restricted to the relevant module.

ie. tox -e py27 openpyxl/reader does not fail but tox -e py27 openpyxl does. The failures are consistent across Python versions.

I suspect that this is related to #228 and will try to create a workaround.

[Themanwithoutaplan]

A workaround is to set warnings.filter('always') somewhere in a test module so that it is executed at runtime. It doesn't work for me if I do it within a test. Does this mean that tox is setting warnings.filters itself?

[nicoddemus]

recwarn itself sets a new filter right before executing the test: https://github.com/pytest-dev/pytest/blob/master/_pytest/recwarn.py#L16

Not sure if this is a problem though.

[nicoddemus]

@Themanwithoutaplan, could you please try with the latest master? I think #922 addressed this issue.

[Themanwithoutaplan]

Sorry for taking such a long time to review this but I still have this problem.

[sloria]

I am also seeing this problem. recwarn and assert_warns both fail to capture warnings on py27, py26, and py33. They work as expected on py34 and py35 only.

https://travis-ci.org/sloria/webargs/builds/89863868

[seberg]

There is a python issue which has been fixed/made better in py34 (the warning context stuff mentioned somewhere here too I think). That will be the reason, nothing pytest can fix, the best you could do it provide a bit more powerful tools, but basically as soon as someone uses anything but "always" and "error" filters, you can run into this. You could give tools (wrote to the list about that) to make it easier to replace "ignore" or "default" filters, but basically, the only way to get around such issues is to make the test very clean.

[Zac-HD]

Fixed by #4104.

[Themanwithoutaplan]

I'm still seeing failures in Python 2.7 even when using a checkout of the current master.

[nicoddemus]

Thanks @Themanwithoutaplan, reopening then.

[Themanwithoutaplan]

I have reliable failures when running tox -e py27 on my project. My current workaround (always warn) is fine for me but causes problems for some downstream projects.

[asottile]

hello, first off thank you for the issue!

python 2.x support has ended for pytest core.

we've decided in #7296 to close the python-2-specific issues to free up some space in our backlog. however, in accordance to our python 2.7 and 3.4 support community patches will still be accepted to the 4.6 series to fix bugs for python 2. (so if this issue is important to you and you have a patch to fix it, feel free to make a PR targeting the 4.6.x branch despite this ticket being closed).