Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FileNotFoundError while writing results against a test suite that modifies current working directory #824

Closed
simonw opened this issue Jul 14, 2019 · 6 comments
Closed

FileNotFoundError while writing results against a test suite that modifies current working directory #824

simonw opened this issue Jul 14, 2019 · 6 comments
Labels
bug Something isn't working

Comments

@simonw
Copy link
Contributor

simonw commented Jul 14, 2019

Describe the bug

When running 5.0a5 against my datasette project I get the following error at the end of the test run:

tests/test_publish_now.py .......                                                                                        [ 81%]
tests/test_utils.py .................................................................                                    [100%]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File ".../site-packages/_pytest/main.py", line 213, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File ".../site-packages/_pytest/main.py", line 257, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File ".../site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File ".../site-packages/pluggy/manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File ".../site-packages/pluggy/manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File ".../site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File ".../site-packages/pytest_cov/plugin.py", line 229, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File ".../site-packages/pytest_cov/engine.py", line 171, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File ".../site-packages/coverage/control.py", line 553, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>   File ".../site-packages/coverage/control.py", line 607, in get_data
INTERNALERROR>     if self._collector and self._collector.flush_data():
INTERNALERROR>   File ".../site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR>     self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 245, in add_lines
INTERNALERROR>     self._choose_lines_or_arcs(lines=True)
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 288, in _choose_lines_or_arcs
INTERNALERROR>     with self._connect() as con:
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 161, in _connect
INTERNALERROR>     self._create_db()
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 118, in _create_db
INTERNALERROR>     with self._dbs[get_thread_id()] as db:
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 616, in __enter__
INTERNALERROR>     self.connect()
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 601, in connect
INTERNALERROR>     filename = os.path.relpath(self.filename)
INTERNALERROR>   File ".../lib/python3.7/posixpath.py", line 475, in relpath
INTERNALERROR>     start_list = [x for x in abspath(start).split(sep) if x]
INTERNALERROR>   File ".../lib/python3.7/posixpath.py", line 383, in abspath
INTERNALERROR>     cwd = os.getcwd()
INTERNALERROR> FileNotFoundError: [Errno 2] No such file or directory

=========== 355 passed in 47.88 seconds =========

I'm pretty sure this is because some of my tests in that test suite mess around with the current working directory. It would be great if coverage.py was robust against changes made to the working directory.

To Reproduce

These steps to reproduce work on my OS X laptop running Python 3.7:

cd /tmp
mkdir coverage-error
cd coverage-error/
pipenv shell
git clone https://github.com/simonw/datasette
cd datasette/
pip install -e .[test]
pip install coverage==5.0a5 pytest-cov
pytest --cov=datasette
@simonw simonw added the bug Something isn't working label Jul 14, 2019
@simonw simonw changed the title FileNotFoundError while writing SQLite database in 5.0a5 FileNotFoundError while writing SQLite database in 5.0a5 against a test suite that modifies current working directory Jul 14, 2019
@simonw
Copy link
Contributor Author

simonw commented Jul 14, 2019

I'm not entirely sure, but I'm suspicious that the test behaviour which triggers this is my use of the Click .isolated_filesystem() testing mechanism: http://click.palletsprojects.com/en/5.x/testing/#file-system-isolation

I use that here (and in a bunch of other places): https://github.com/simonw/datasette/blob/a9453c4dda70bbf5122835e68f63db6ecbe1a6fc/tests/test_publish_now.py#L118-L119

@simonw
Copy link
Contributor Author

simonw commented Jul 14, 2019

It looks like this bug is NOT unique to the 5.0 series.

I ran pip install coverage==4.5.3 in that same testing environment and then ran pytest --cov=datasette again and got this, similar error:

INTERNALERROR>   File ".../site-packages/pytest_cov/plugin.py", line 229, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File ".../site-packages/pytest_cov/engine.py", line 171, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File ".../site-packages/coverage/control.py", line 675, in load
INTERNALERROR>     self._init()
INTERNALERROR>   File ".../site-packages/coverage/control.py", line 223, in _init
INTERNALERROR>     set_relative_directory()
INTERNALERROR>   File ".../site-packages/coverage/files.py", line 28, in set_relative_directory
INTERNALERROR>     RELATIVE_DIR = os.path.normcase(abs_file(os.curdir) + os.sep)
INTERNALERROR>   File ".../site-packages/coverage/files.py", line 163, in abs_file
INTERNALERROR>     path = os.path.realpath(path)
INTERNALERROR>   File ".../lib/python3.7/posixpath.py", line 396, in realpath
INTERNALERROR>     return abspath(path)
INTERNALERROR>   File ".../lib/python3.7/posixpath.py", line 383, in abspath
INTERNALERROR>     cwd = os.getcwd()
INTERNALERROR> FileNotFoundError: [Errno 2] No such file or directory

@simonw simonw changed the title FileNotFoundError while writing SQLite database in 5.0a5 against a test suite that modifies current working directory FileNotFoundError while writing results against a test suite that modifies current working directory Jul 14, 2019
@nedbat
Copy link
Owner

nedbat commented Jul 14, 2019

Something is in the air... this was fixed on master last week: #806 . I have no idea why it hasn't been reported before.

@nedbat nedbat closed this as completed Jul 14, 2019
@simonw
Copy link
Contributor Author

simonw commented Jul 15, 2019

Unfortunately I don't think it's fixed in master - I tried running against master by using the following:

pip install https://github.com/nedbat/coveragepy/archive/master.zip

Then running pytest --cov=datasette again... but I still got the same error:

datasette/tests/test_utils.py .................................................................                                                                             [100%]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File ".../site-packages/_pytest/main.py", line 213, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File ".../site-packages/_pytest/main.py", line 257, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File ".../site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File ".../site-packages/pluggy/manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File ".../site-packages/pluggy/manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File ".../site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File ".../site-packages/pytest_cov/plugin.py", line 229, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File ".../site-packages/pytest_cov/engine.py", line 171, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File ".../site-packages/coverage/control.py", line 575, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>   File ".../site-packages/coverage/control.py", line 629, in get_data
INTERNALERROR>     if self._collector and self._collector.flush_data():
INTERNALERROR>   File ".../site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR>     self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 320, in add_lines
INTERNALERROR>     self._choose_lines_or_arcs(lines=True)
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 368, in _choose_lines_or_arcs
INTERNALERROR>     with self._connect() as con:
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 236, in _connect
INTERNALERROR>     self._create_db()
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 192, in _create_db
INTERNALERROR>     with self._dbs[get_thread_id()] as db:
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 778, in __enter__
INTERNALERROR>     self.connect()
INTERNALERROR>   File ".../site-packages/coverage/sqldata.py", line 756, in connect
INTERNALERROR>     filename = os.path.relpath(self.filename)
INTERNALERROR>   File ".../lib/python3.7/posixpath.py", line 475, in relpath
INTERNALERROR>     start_list = [x for x in abspath(start).split(sep) if x]
INTERNALERROR>   File ".../lib/python3.7/posixpath.py", line 383, in abspath
INTERNALERROR>     cwd = os.getcwd()
INTERNALERROR> FileNotFoundError: [Errno 2] No such file or directory

@simonw
Copy link
Contributor Author

simonw commented Jul 15, 2019

Maybe this is a bug in pytest-cov now?

@simonw
Copy link
Contributor Author

simonw commented Jul 15, 2019

Aha - maybe it's this bug over there: pytest-dev/pytest-cov#306

@ogrisel ogrisel mentioned this issue Dec 18, 2019
Closed
@simonw simonw mentioned this issue Jun 5, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@simonw @nedbat