Call "--collect-only" programmatically

[westhomas]

I need to get a list of tests that py.test can see in my code.

I can see session.perform_collect() here:

pytest/_pytest/main.py

Lines 138 to 139 in 8639bf7

	def pytest_collection(session):
	return session.perform_collect()

But I'm not sure how to get a session or the config necessary to create a session.

What's the sequence of commands to be able to call the perform_collect and then parse it as a python dict?

[nicoddemus]

I think the easiest way is by writing a plugin and invoking pytest.main() while enabling that plugin. This way you have the entire hook facility available to you.

# contents of runpytest.py
import sys
import pytest

class MyPlugin:

    def __init__(self):
        self.collected = []

    def pytest_collection_modifyitems(self, items):
        for item in items:
            self.collected.append(item.nodeid)


my_plugin = MyPlugin()
directory = sys.argv[1]
pytest.main(['--collect-only', directory], plugins=[my_plugin])

for nodeid in my_plugin.collected:
    print(nodeid)

Running python runpytest.py test_foo.py prints:

============================= test session starts =============================
platform win32 -- Python 3.5.0, pytest-3.0.4.dev, py-1.4.31, pluggy-0.4.0
rootdir: C:\pytest, inifile: tox.ini
plugins: hypothesis-3.5.0
collected 10 items
<Module '.tmp/test_foo.py'>
  <Function 'test_foo[0]'>
  <Function 'test_foo[1]'>
  <Function 'test_foo[2]'>
  <Function 'test_foo[3]'>
  <Function 'test_foo[4]'>
  <Function 'test_foo[5]'>
  <Function 'test_foo[6]'>
  <Function 'test_foo[7]'>
  <Function 'test_foo[8]'>
  <Function 'test_foo[9]'>

======================== no tests ran in 0.01 seconds =========================
.tmp/test_foo.py::test_foo[0]
.tmp/test_foo.py::test_foo[1]
.tmp/test_foo.py::test_foo[2]
.tmp/test_foo.py::test_foo[3]
.tmp/test_foo.py::test_foo[4]
.tmp/test_foo.py::test_foo[5]
.tmp/test_foo.py::test_foo[6]
.tmp/test_foo.py::test_foo[7]
.tmp/test_foo.py::test_foo[8]
.tmp/test_foo.py::test_foo[9]

If you don't want any pytest output, you can disable the terminal plugin:

pytest.main(['--collect-only', '-p', 'no:terminal', directory], plugins=[my_plugin])

$ python runpytest.py test_foo.py
.tmp/test_foo.py::test_foo[0]
.tmp/test_foo.py::test_foo[1]
.tmp/test_foo.py::test_foo[2]
.tmp/test_foo.py::test_foo[3]
.tmp/test_foo.py::test_foo[4]
.tmp/test_foo.py::test_foo[5]
.tmp/test_foo.py::test_foo[6]
.tmp/test_foo.py::test_foo[7]
.tmp/test_foo.py::test_foo[8]
.tmp/test_foo.py::test_foo[9]

[yangyunguo]

Thanks! It works!

[westhomas]

Perfect! Thanks!

[timgkeller]

Is this still considered the way to go?

[RonnyPfannschmidt]

Yes

[elrandira]

for me, MyPlugin.pytest_collection_modifyitems() was never called by Pytest I had to replace it with a MyPlugin.pytest_collection_finish method. It worked very well :)

[RonnyPfannschmidt]

That sound like a bug, can you elaborate?

[elrandira]

I will try to reproduce.

My employer uses a complex "package" of multiple scripts, It changes the import depending on the configuration and use multiple conftest.py and plugins. I was wondering if some plugisn or conftest might have prevent my pytest_collection_modifyitems from being called.

[RonnyPfannschmidt]

That strongly depends on how your plugin is added

It's possibly a bug in pytest that the hook is called at all i. Case it comes in from a conftest

[elrandira]

well, I couldn't reproduce today. It correctly entered pytest_collection_modifyitems :/

for context, this is a part of my pip list:
pytest 7.1.3
pytest-cov 3.0.0
pytest-durations 1.2.0
pytest-forked 1.4.0
pytest-json-report 1.5.0
pytest-metadata 2.0.4
pytest-mock 3.6.1
pytest-parallel 0.1.1
pytest-progress 1.2.5
pytest-random-order 1.0.4
pytest-timeout 1.4.2
pytest-xdist 1.29.0