Fix windows #157

nicoddemus · 2017-10-25T22:07:51Z

Trying to make the Windows CI work again.

syslog is not available on Windows

nicoddemus · 2017-10-26T00:07:36Z

Had to skip a lot of tests on Windows to get the suite to pass: #161 and #162. Also skipping one of the tests which was failing under xdist (#160).

I want to have the suite passing on Windows specially for the parts which are still used by pytest: for example, the "x" flag being passed to py.path would break pytest on Windows immediately upon release.

…Python versions

RonnyPfannschmidt · 2017-10-26T05:21:13Z

py/_path/local.py

        def create_lockfile(path):
            """ exclusively create lockfile. Throws when failed """
            mypid = os.getpid()
            lockfile = path.join('.lock')
            if hasattr(lockfile, 'mksymlinkto'):
                lockfile.mksymlinkto(str(mypid))
            else:
-                lockfile.write(str(mypid), 'wx')
+                lockfile.write(str(mypid), 'w' + exclusive_flag)


as far as i can tell without the exclusive flag the lockfile creation is broken

Not sure what to do here, because passing wx to lockfile.write raises an exception on Windows:

ValueError: must have exactly one of create/read/write/append mode

Also, it is problematic then that removing this flag didn't break any test. Either it is not necessary or we are not covering the case where the flag is needed.

the x flag is absolutely critical as it ensures exclusive open - else the operation wont fail if the element exists

the x flag is absolutely critical as it ensures exclusive open - else the operation wont fail if the element exists

I got that. 😄 It is still a problem that no test breaks on Windows after the flag has been removed.

Just to be clear: Windows does not support the x flag so we can't publish 1.5 with this change otherwise we will break every new pytest installation where the test suite uses tmpdir.

How should we proceed?

i ccd you in the relevant pr

Thanks! Let's move the discussion there then.

RonnyPfannschmidt · 2017-10-26T05:22:15Z

testing/path/test_svnwc.py

@@ -5,6 +5,13 @@
 from py._path import svnwc as svncommon
 from svntestbase import CommonSvnTests

+
+@pytest.fixture(autouse=True)
+def skip_on_windows():


just use a mark?

Duh don't know what I was thinking. Thanks!

Related to pytest-dev#161

Due to known mark transfer problems, the global pytestmark ends up overwriting the mark which was originally applied to this test

RonnyPfannschmidt · 2017-10-27T12:30:31Z

testing/path/test_svnwc.py

    def test_status_update(self, path1):
+        # not a mark because the global "pytestmark" will end up overwriting a mark here
+        pytest.xfail("svn-1.7 has buggy 'status --xml' output")


this is fixed in recent pytest and should be in the next feature release (the evaluator refactoring)

@aurzenligl

Thanks to @aurzenligl for the solution

nicoddemus · 2017-11-05T00:17:47Z

Pushed a new commit with the file locking fix for Windows as suggested by @aurzenligl in #143, kudos to him. 👍

nicoddemus · 2017-11-05T00:20:29Z

testing/path/test_local.py

@@ -446,13 +453,6 @@ def notimpl(x, y):
        assert x.relto(tmpdir)
        assert x.check()

-    def test_make_numbered_dir_multiprocess_safe(self, tmpdir):


The reason why this test was not catching the regression in the previous locking code for Windows is that it was being skipped on Windows (this class has the pytestmark = skiponwin32 mark).

RonnyPfannschmidt · 2017-11-05T09:45:09Z

py/_path/local.py

        def create_lockfile(path):
            """ exclusively create lockfile. Throws when failed """
            mypid = os.getpid()
            lockfile = path.join('.lock')
            if hasattr(lockfile, 'mksymlinkto'):
                lockfile.mksymlinkto(str(mypid))
            else:
-                lockfile.write(str(mypid), 'w' + exclusive_flag)
+                try:
+                    fd = os.open(str(lockfile), os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o644)


use py.error.checked_call here

main reason is to account for all the other errors that could unexpectedly happen

nicoddemus · 2017-11-09T17:02:28Z

I'm hitting this bug when running on Python 2.6 on Windows:

AssertionError    prepare(preparation_data)
  File "D:\Programming\Python26\Lib\multiprocessing\forking.py", line 449, in prepare
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\Programming\Python26\Lib\multiprocessing\forking.py", line 341, in main
:     _assert main_name not in sys.modules, main_name
Traceback (most recent call last):
_main__AssertionError:
  File "<string>", line 1, in <module>
    _prepare(preparation_data)
_main__
  File "D:\Programming\Python26\Lib\multiprocessing\forking.py", line 449, in prepare
  File "D:\Programming\Python26\Lib\multiprocessing\forking.py", line 341, in main
        assert main_name not in sys.modules, main_name
prepare(preparation_data)
AssertionError  File "D:\Programming\Python26\Lib\multiprocessing\forking.py", line 449, in prepare
: _    _main__assert main_name not in sys.modules, main_name

AssertionError: __main__

This is the reason why the py26 environments are all timing out.

I could trace it to this bug which has been fixed in Python 2.7.11. The bug is related to the fact that Windows does not have fork functionality.

My suggestion is to just skip that test in Python 2.6 and Windows for now, given that we plan to remove 2.6 and 3.3 support soon anyway (#159).

RonnyPfannschmidt · 2017-11-09T17:37:06Z

i agree

Remove invalid 'x' flag to file.write on Windows

cf5aa83

nicoddemus requested a review from RonnyPfannschmidt October 25, 2017 22:07

Skip test_log.py tests on Windows because 'syslog' is not available

cd13abe

nicoddemus force-pushed the fix-windows branch from 70448e6 to cd13abe Compare October 25, 2017 22:52

nicoddemus added 5 commits October 25, 2017 21:11

Skip TestSvnWCAuthFunctional.test_switch when executing under xdist

d57aedb

Related to pytest-dev#160

Skip tests in test_py_imports related to py.log on Windows

cd23b8d

syslog is not available on Windows

Skip tests in test_svnwc on Windows

5a2f1ed

Related to pytest-dev#161

Fix skip() call in test_svnauth

f9c5d6a

Skip test_warning on Windows in pytest >= 3.1

5c1a797

Related to pytest-dev#162

Change message check about ZeroDivisionError so it works in multiple …

110bc59

…Python versions

RonnyPfannschmidt requested changes Oct 26, 2017

View reviewed changes

nicoddemus added 3 commits October 26, 2017 18:33

Use an xfail mark instead of a fixture in test_svnwc

1eda5c9

Related to pytest-dev#161

Change fixture to a mark in test_warning

3cfe7c6

Change mark from test_status_update to an imperative xfail

9a808f1

Due to known mark transfer problems, the global pytestmark ends up overwriting the mark which was originally applied to this test

RonnyPfannschmidt reviewed Oct 27, 2017

View reviewed changes

RonnyPfannschmidt mentioned this pull request Oct 27, 2017

make_numbered_dir() multi-process-safe #143

Merged

Fix file locking for tmpdir on Windows

6fc21fa

Thanks to @aurzenligl for the solution

nicoddemus commented Nov 5, 2017

View reviewed changes

RonnyPfannschmidt reviewed Nov 5, 2017

View reviewed changes

Use py.error.checked_call when creating the lock file for tmpdir

ad80946

RonnyPfannschmidt approved these changes Nov 9, 2017

View reviewed changes

nicoddemus added 2 commits November 9, 2017 15:45

Skip multiprocess tmpdir creation safety test on Windows/Python 2.6

bdc9112

Add py.error.EBUSY to the list of possible errors when creating locks

2d30cde

nicoddemus merged commit e5f9547 into pytest-dev:master Nov 9, 2017

nicoddemus deleted the fix-windows branch November 9, 2017 20:25

nicoddemus mentioned this pull request Nov 9, 2017

Drop support for EOL Python <= 2.6, 3.0-3.3 #165

Merged

This was referenced Nov 15, 2017

py 1.5 breaks py.log: "No module named syslog" #170

Closed

py 1.5 breaks freezegun on Windows #169

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix windows #157

Fix windows #157

nicoddemus commented Oct 25, 2017

nicoddemus commented Oct 26, 2017

RonnyPfannschmidt Oct 26, 2017

nicoddemus Oct 26, 2017

nicoddemus Oct 27, 2017

RonnyPfannschmidt Oct 27, 2017

nicoddemus Oct 27, 2017

RonnyPfannschmidt Oct 27, 2017

nicoddemus Oct 27, 2017

RonnyPfannschmidt Oct 26, 2017

nicoddemus Oct 26, 2017

RonnyPfannschmidt Oct 27, 2017

nicoddemus commented Nov 5, 2017

nicoddemus Nov 5, 2017 •

edited

Loading

RonnyPfannschmidt Nov 5, 2017

RonnyPfannschmidt Nov 5, 2017

nicoddemus commented Nov 9, 2017

RonnyPfannschmidt commented Nov 9, 2017

Fix windows #157

Fix windows #157

Conversation

nicoddemus commented Oct 25, 2017

nicoddemus commented Oct 26, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicoddemus commented Nov 5, 2017

nicoddemus Nov 5, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicoddemus commented Nov 9, 2017

RonnyPfannschmidt commented Nov 9, 2017

nicoddemus Nov 5, 2017 •

edited

Loading