pytest_addoption - Command-line options not added

[pytestbot]

Originally reported by: Jason R. Coombs (BitBucket: jaraco, GitHub: jaraco)

---

I created a sample minimal project (attached). In myproj/test/conftest.py you find:

def pytest_addoption(parser):
    raise ValueError("got here")
    parser.addoption('--super-option', action="store_true", default=False)

The ValueError was added to quickly determine if pytest is processing the options. Run pytest on the project and it quickly returns:

myproj\test\conftest.py:2: in pytest_addoption
>       raise ValueError("got here")
E    ValueError: got here
=========================== 1 error in 0.08 seconds ===========================

However, if one passes a command-line argument (even --help), the pytest_addoption is never executed:

> py.test --help
Usage: py.test-script.py [options] [file_or_dir] [file_or_dir] [...]
...
[nothing about super-option]

Preferably, pytest would process pytest_addoption even when options are supplied. If it can't do this for some reason, it should at least be documented where pytest_addoption is valid.

This behavior was observed on pytest 2.0.3 and 2.1.2 on both Ubuntu Lucid and Windows 7 on Python 2.7 64-bit.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/78

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

Are you running the two example commands from different current working directories? pytest does not recurse into subdirectories to find conftest.py files.

[pytestbot]

Original comment by Jason R. Coombs (BitBucket: jaraco, GitHub: jaraco):

---

I'm confused. In every project I use, pytest does discover conftest.py files in subdirectories (and has for a long time). Our tests wouldn't run if it didn't find conftest.py in subdirectories.

The two example tests are running from the same directory (the parent of the project). I extracted the zip file to my desktop and here is the full, unedited transcript showing what happens with and without command-line options supplied.

{{{
PS C:\Users\jaraco> cd .\Desktop
PS C:\Users\jaraco\Desktop> py.test
============================= test session starts =============================
platform win32 -- Python 2.7.2 -- pytest-2.1.2
collected 0 items / 1 errors

=================================== ERRORS ====================================
_____________________________ ERROR collecting . ______________________________
C:\python\lib\site-packages\py-1.4.5-py2.7.egg\py_path\common.py:312: in visit

  for x in Visitor(fil, rec, ignore, bf, sort).gen(self):

C:\python\lib\site-packages\py-1.4.5-py2.7.egg\py_path\common.py:358: in gen
for p in self.gen(subdir):
C:\python\lib\site-packages\py-1.4.5-py2.7.egg\py_path\common.py:348: in gen
if p.check(dir=1) and (rec is None or rec(p))])
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\main.py:468: in _recurse
ihook.pytest_collect_directory(path=path, parent=self)
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\main.py:126: in call_matching_hooks
plugins = self.config._getmatchingplugins(self.fspath)
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\config.py:281: in _getmatchingplugins
plugins += self._conftest.getconftestmodules(fspath)
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\config.py:183: in getconftestmodules
clist.append(self.importconftest(conftestpath))
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\config.py:219: in importconftest
self._postimport(mod)
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\config.py:224: in _postimport
self._onimport(mod)
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\config.py:270: in _onimportconftest
self.pluginmanager.consider_conftest(conftestmodule)
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\core.py:179: in consider_conftest
if self.register(conftestmodule, name=conftestmodule.file):
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\core.py:89: in register
self.hook.pytest_plugin_registered(manager=self, plugin=plugin)
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\core.py:411: in call
return self._docall(methods, kwargs)
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\core.py:422: in _docall
res = mc.execute()
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\core.py:340: in execute
res = method(**kwargs)
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\core.py:221: in pytest_plugin_registered
{'parser': self._config._parser})
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\core.py:309: in call_plugin
kwargs=kwargs, firstresult=True).execute()
C:\python\lib\site-packages\pytest-2.1.2-py2.7.egg_pytest\core.py:340: in execute
res = method(**kwargs)
myproj\test\conftest.py:2: in pytest_addoption
raise ValueError("got here")
E ValueError: got here
=========================== 1 error in 0.07 seconds ===========================
PS C:\Users\jaraco\Desktop> py.test --help
Usage: py.test-script.py [options] [file_or_dir] [file_or_dir] [...]

Options:
general:
-k KEYWORDEXPR only run tests which match given keyword expression.
An expression consists of space-separated terms. Each
term must match. Precede a term with '-' to negate.
Terminate expression with ':' to make the first match
match all subsequent tests (usually file-order).
-x, --exitfirst exit instantly on first error or failed test.
--maxfail=num exit after first num failures or errors.
--funcargs show available function arguments, sorted by plugin
--pdb start the interactive Python debugger on errors.
--capture=method per-test capturing method: one of fd (default)|sys|no.
-s shortcut for --capture=no.
--runxfail run tests even if they are marked xfail

reporting:
-v, --verbose increase verbosity.
-q, --quiet decreate verbosity.
-r chars show extra test summary info as specified by chars
(f)ailed, (s)skipped, (x)failed, (X)passed.
-l, --showlocals show locals in tracebacks (disabled by default).
--report=opts (deprecated, use -r)
--tb=style traceback print mode (long/short/line/native/no).
--fulltrace don't cut any tracebacks (default is to cut).
--pastebin=mode send failed|all info to Pocoo pastebin service.
--junitxml=path create junit-xml style report file at given path.
--junitprefix=str prepend prefix to classnames in junit-xml output
--resultlog=path path for machine-readable result log.

collection:
--collectonly only collect tests, don't execute them.
--pyargs try to interpret all arguments as python packages.
--ignore=path ignore path during collection (multi-allowed).
--confcutdir=dir only load conftest.py's relative to specified dir.
--doctest-modules run doctests in all .py modules
--doctest-glob=pat doctests file matching pattern, default: test*.txt

test session debugging and configuration:
--basetemp=dir base temporary directory for this test run.
--version display pytest lib version and import information.
-h, --help show help message and configuration info
-p name early-load given plugin (multi-allowed).
--traceconfig trace considerations of conftest.py files.
--debug store internal tracing debug information in
'pytestdebug.log'.
--assert=MODE control assertion debugging tools. 'plain' performs no
assertion debugging. 'reinterp' reinterprets assert
statements after they failed to provide assertion
expression information. 'rewrite' (the default)
rewrites assert statements in test modules on import
to provide assert expression information.
--no-assert DEPRECATED equivalent to --assert=plain
--nomagic DEPRECATED equivalent to --assert=plain
--genscript=path create standalone py.test script at given target path.

[pytest] ini-options in the next pytest.ini|tox.ini|setup.cfg file:

addopts (args) extra command line options
minversion (string) minimally required pytest version
norecursedirs (args) directory patterns to avoid for recursion
python_files (args) glob-style file patterns for Python test module disc
python_classes (args) prefixes for Python test class discovery
python_functions (args) prefixes for Python test function and method discove
}}}

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

Here is what happens:

During "py.test" the sub directories are traversed for collecting test files. When a conftest.py files is encountered its addoption and configure hooks are executed. This makes default option values available to any test in a subdirectory.

However, during a "py.test --help" no sub directories are traversed and no tests are collected. Making "py.test --help" traverse sub directories is not a good idea at least for large projects.

Maybe it makes sense to introduce a mechanism for preloading conftest.py files using the config file mechanism. It already mostly works if you put this into a pytest.ini (or tox.ini or setup.cfg):

{{{
[pytest]
addopts = -p myproj.test.conftest
}}}

This means that in your project you get the command line options everywhere.

Note that you need to also add a myproj/test/init.py file for this to work. Maybe we could allow filenames so that this requirement is not there.

What do you think?

[pytestbot]

Original comment by Jason R. Coombs (BitBucket: jaraco, GitHub: jaraco):

---

Oh. That's very interesting. I understand better how it works, and that all makes sense. I think I was assuming that since the add_option was not being called during --help that pytest would never have included that option.

Are you saying that pytest will honor those command-line arguments if they're passed, even though they don't appear in --help? Or are you saying that they will be defined their defaults, but to supply those arguments, the tests need to be run from that directory? In my findings, it's the latter. I could not find any way to get the command-line options to be included unless they were processed as the top-level config. I tried using {{{-p myproj.test.conftest}}} (after touching myproj/test/init.py), but that gave me an error:

{{{
PS C:\Users\jaraco\desktop\myproj> py.test -p myproj.test.conftest --help
Traceback (most recent call last):
File "c:\python\scripts\py.test-script.py", line 9, in
load_entry_point('pytest==2.2.1', 'console_scripts', 'py.test')()
File "C:\Program Files\Python27\lib\site-packages\pytest-2.2.1-py2.7.egg_pytest\core.py", line 467, in main
config = _prepareconfig(args, plugins)
...
File "C:\Program Files\Python27\lib\site-packages\pytest-2.2.1-py2.7.egg_pytest\core.py", line 196, in import_plugin
mod = importplugin(modname)
File "C:\Program Files\Python27\lib\site-packages\pytest-2.2.1-py2.7.egg_pytest\core.py", line 330, in importplugin
return import(importspec, None, None, 'doc')
ImportError: No module named myproj.test.conftest
}}}
{{{
PS C:\Users\jaraco\desktop\myproj> python -c "import myproj.test.conftest"

the above command produces no output because it succeeds.

}}}

Am I not using -p correctly? Apparently if I set {{{PYTHONPATH='.'}}}, it works as prescribed.

After going through all of this, my feeling is it would be sufficient to simply document that command-line options should be defined at the level where it's expected py.test will be run or explicitly loaded using -p. Whether -p supports file-based loading is perhaps the subject for another ticket (though I don't feel strongly enough to create one).

Now that I understand better, I'm going to downgrade the priority of this ticket. The explanation was most helpful, but I would suggest adding a note in the docs.

Thanks!

[nicoddemus]

The docs now contain this explanation, so closing this.