Add a CLI option for verbosity to the pytest plugin

[Zac-HD]

The HYPOTHESIS_VERBOSITY_LEVEL environment variable is now deprecated (from #1211). An easy way to control verbosity is still useful though, so we would like to support this as a command-line flag.

This would be implemented in hypothesis.extra.pytestplugin, similarly to the deprecated version here. The new --hypothesis-verbosity option should be applied after loading the profile specified by --hypothesis-profile (if given).

Finally, the new option should be listed in docs/details.rst, including that the verbosity option is applied after loading a profile.

If you would like to work on this issue, feel free to comment and I will help you get started!

[pawling11]

Hi Zac. I think I would be interested in helping out with this issue.
From a quick glance, looks like I would need to add a constant in pytestplugin.py and add some code to the pytest_addoption and pytest_configure functions to handle verbosity.

I presume the deprecated code in _settings.py would need removed? However, I am new to this project and not very familiar with how the Verbosity class is used. Do the class methods need to be retained?

Thanks in advance!

[Zac-HD]

Hi Zac. I think I would be interested in helping out with this issue.

Awesome! We (and I) are always delighted to meet new contributors 😄
(and again, just let me know if you have any questions or want a hand with something)

I presume the deprecated code in _settings.py would need removed?

We'll actually keep it around for a while - Semantic Versioning means that it will not be an error before Hypothesis 4.0 (later this year), and we also have a stability policy that says we should wait six months before removing a deprecated feature, which might push the final removal to 5.0. That can feel like a very long time as a dev, but it's great as a user!

From a quick glance, looks like I would need to add a constant in pytestplugin.py and add some code to the pytest_addoption and pytest_configure functions to handle verbosity.

Exactly right! I linked to _settings.py as the example code to copy for pytest_addoption, so that should be easy enough.
It would be great to document these options as part of the same pull request, but if you'd rather not that can happen later.

[you didn't ask for general advice, but here it is:]

• I suggest opening a pull request while it's still a work-in-progress - mine always are, even when I didn't plan it that way
• Read our documentation guide, especially the changelog section
• Add yourself to the list of contributors in CONTRIBUTING.rst

[pawling11]

Good to know about the semantic versioning policy used in this project. I will leave the code in _settings.py untouched. Thanks for the advice too, I will look over the documentation guide before making my contribution. I will probably need to brush up on github real quick too, haha.

Exactly right! I linked to _settings.py as the example code to copy for pytest_addoption, so that should be easy enough.

Cool! Yes this part appears pretty straightforward.

I would not be opposed to writing some documentation. If I understand correctly, the available parameters for this option for this are:
quiet = 0
normal = 1
verbose = 2
debug = 3

A few quick questions.

• Do you have any recommendations for where to look that would give me a better idea of what each respective option does?
• Should there be an action parameter passed to the group.addoption call for the verbosity option?
• If a default option is provided (I'm guessing one should be), is any particular naming convention preferred (e.g. default= 'normal' or default='1')?

[Zac-HD]

Valid options for the setting are the members of the Verbosity enum. Valid options for the pytest flag are the names, e.g. --hypothesis-verbosity=debug (because we'll then use the string to look up the enum member). The enum module docs are a good reference if you're curious.

Some docs on verbosity here, though having found them they're incomplete (also controls debug logging, deprecation warnings, etc.). I suggest linking to the settings page instead of re-describing the behaviour of each option, and I'll fix that end!

Yes, use action='store' to save the string for later use.

There should not be a default option - instead, treat it like SEED in pytest_configure and only set the verbosity if the option was passed.

[BexDunn]

I'm having a look at this one @ pyconAU sprintday

[BexDunn]

Here's my diff:
master...BexDunn:bextemp

got to head off but will keep working on this over the next few days.

[Zac-HD]

@BexDunn - how is this going? No pressure, just let me know if I can help 😄

[BexDunn]

@Zac-HD tonight is the first time I've actually managed to sit down and check this issue, sorry!

[Zac-HD]

No worries! Ping me if you want a review or some tips on how to test this; if I don't hear anything I'll assume you just want to puzzle it out yourself 😄

(though if the issue is inactive for more than a week someone else might finish it (and credit you) - so check in occasionally if you want to do it all yourself!)