Fix skipped tests on Windows

[jherland]

Fix tests that were skipped on Windows until now.

These were not skipped due to lack of Windows compatibility in FalwtyDeps itself, but rather due to lack of compatibility (i.e. latent POSIX assumptions) in our test framework.

[jherland]

Crap, while trying to fix test_real_projects on Windows I've run into an issue that is not strictly Windows-specific (although it only show up on Windows here), and that will require more thinking/changes in our core logic. (@mknorps: I suspect you might have run into the same issue while working on the initial Windows support). Here it is:

Tensorflow is weird on Windows: On other systems, it's "just" a large package (several hundred MBs), but on Windows, it is a very small (1.9kB) placeholder package that does not actually provide a tensorflow import name at all, rather it depends on another package, tensorflow-intel, which contains the actual meat (266MB). Since we pip install with --no-deps, we would need to explicitly include that package in our test requirements order to make the tensorflow import name available on Windows.

Howver, this is not enough: Our analysis would still see that the tensorflow package does not proved the tensorflow import name, and tensorflow ends up BOTH as undeclared AND unused!

To fix this, I think FawltyDeps needs to somehow realize the dependency relationship between the placeholder package (tensorflow) and its dependency (tensorflow-intel), and somehow attribute the latter's import names to the former.

This is hard, because a lot of packages have dependencies that are unrelated to the import name they are "expected" to provide, so we might have to make a corner case/exception for "placeholder packages" here... More thinking needed...

---

After some more thinking I realize I've seen this issue before. It is pretty much exactly the same as #176!

Anyway, I don't plan to fix this deeper issue as part of this PR. We will need to work around this issue for the time being (probably simply skip this test on Windows).

[mknorps]

Results of tests on Windows.

Before:

=================================================================== short test summary info ====================================================================
FAILED tests/test_gitignore_parser.py::test_symlink_to_another_directory - OSError: [WinError 1314] A required privilege is not held by the client: 'C:\\Users\\ttyle\\AppData\\Local\\Temp\\pytest-of-ttyle\\pytest-160\\test_symlink_...
===================================== 1 failed, 1079 passed, 146 skipped, 42 deselected, 77 warnings in 134.14s (0:02:14) ======================================

After:

=================================================================== short test summary info ====================================================================
FAILED tests/test_gitignore_parser.py::test_symlink_to_another_directory - OSError: [WinError 1314] A required privilege is not held by the client: 'C:\\Users\\ttyle\\AppData\\Local\\Temp\\pytest-of-ttyle\\pytest-161\\test_symlink_...
====================================== 1 failed, 1193 passed, 60 skipped, 48 deselected, 85 warnings in 135.54s (0:02:15) ======================================

There is one test that is failing due to symlink, but let's skip it for now as it was the problem also in my "vanilla" run.

[mknorps]

I include the encountered failure for the record here:

=========================================================================== FAILURES ===========================================================================
______________________________________________________________ test_symlink_to_another_directory _______________________________________________________________

tmp_path = WindowsPath('C:/Users/ttyle/AppData/Local/Temp/pytest-of-ttyle/pytest-161/test_symlink_to_another_direct0')

    def test_symlink_to_another_directory(tmp_path):
        project_dir = tmp_path / "project_dir"
        link = project_dir / "link"
        target = tmp_path / "another_dir/target"

        project_dir.mkdir(parents=True, exist_ok=True)
>       link.symlink_to(target)

tests\test_gitignore_parser.py:240:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = WindowsPath('C:/Users/ttyle/AppData/Local/Temp/pytest-of-ttyle/pytest-161/test_symlink_to_another_direct0/project_dir/link')
target = WindowsPath('C:/Users/ttyle/AppData/Local/Temp/pytest-of-ttyle/pytest-161/test_symlink_to_another_direct0/another_dir/target')
target_is_directory = False

    def symlink_to(self, target, target_is_directory=False):
        """
        Make this path a symlink pointing to the target path.
        Note the order of arguments (link, target) is the reverse of os.symlink.
        """
        if not hasattr(os, "symlink"):
            raise NotImplementedError("os.symlink() not available on this system")
>       os.symlink(target, self, target_is_directory)
E       OSError: [WinError 1314] A required privilege is not held by the client: 'C:\\Users\\ttyle\\AppData\\Local\\Temp\\pytest-of-ttyle\\pytest-161\\test_symlink_to_another_direct0\\another_dir\\target' -> 'C:\\Users\\ttyle\\AppData\\Local\\Temp\\pytest-of-ttyle\\pytest-161\\test_symlink_to_another_direct0\\project_dir\\link'

..\..\..\AppData\Local\Programs\Python\Python312\Lib\pathlib.py:1387: OSError
======================================================================= warnings summary =======================================================================
fawltydeps\extract_declared_dependencies.py:16
  C:\Users\ttyle\programming\FawltyDeps2\FawltyDeps\fawltydeps\extract_declared_dependencies.py:16: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import Requirement

tests/test_cmdline.py: 5 warnings
tests/test_cmdline_options.py: 17 warnings
tests/test_extract_declared_dependencies_errors.py: 12 warnings
tests/test_extract_declared_dependencies_success.py: 50 warnings
  C:\Users\ttyle\programming\FawltyDeps2\FawltyDeps\fawltydeps\limited_eval.py:86: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
    if isinstance(node, (ast.Constant, ast.Str)):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

[mknorps]

Awesome 🎉

It must have took a lot of work to go through all those tests and fix them. Thank you @jherland !

venv_commands function is a good improvement. I really liked how you utilized touch from PathLib and was left with only one line of Windows/Posix distinction as an if statement.

I tested how the updated FawltyDeps works on Windows, and it looks really good 💯

I have one nit-picking comment, that you can act on either by commenting somewhere in BaseExperiment and BaseProject or change the structure of those classes. The latter may be nicer but more time consuming.
The problem is: what if you have both flags posix_only, windows_only in place? Should you skip on both systems or run on both systems?
What if instead of those two flags you had:

systems = ["posix", "windows"]

with default both options and the systems choice will be narrowed down by systems = ["posix]`

[jherland]

The problem is: what if you have both flags posix_only, windows_only in place? Should you skip on both systems or run on both systems?

I did have this check, that would fail the test if both flags were set: https://github.com/tweag/FawltyDeps/pull/412/files#diff-96a370372e6f28b33cfb75c3357c881efb4982beb0b5cebe203869ff85e7ffc6R292

But, really, I agree with you that the data structure representing this should be improved.

What if instead of those two flags you had:

systems = ["posix", "windows"]

with default both options and the systems choice will be narrowed down by systems = ["posix]`

Yes, this is much closer to what we want. Furthermore, we should try to replace the strings with "more proper" enum values. Fortunately there is something for this situation in the standard library: enum.Flag allows you to create an enum and furthermore combine the enum values within the same type!

That is, instead of creating a Compatibility Enum type, and the using a set of {Compatibility.POSIX, Compatibility.WINDOWS} to represent the default/full compatibility, we can create a Compatibility Flag type that uses bitwise OR to combine flags and the combination remains a valid Flag value: Compatibility.POSIX | Compatibility.WINDOWS is a valid value.

e44a405 contains the details. 😄