Windows host base-location for network share not working

[mfgering]

Problem: Windows host, target directory (base-location) cannot be a UNC-style path, e.g. \foo.bar\baz or //foo.bar/baz.
I have tried using the --option base-location=//foo.bar/baz CLI method various different ways with no luck. Using a json config file doesn't work either.

Discovered: The function gallery_dl.path.PathFormat.set_directory (path.py line 179) attempts to adjust for Windows path limits, but it messes up the directory.

For example, if directory is '//foo.bar/baz', then it gets converted to '\?\\foo.bar\baz' and this does not work on Windows 11.

Problem code:

        if WINDOWS:
            # Enable longer-than-260-character paths on Windows
            directory = "\\\\?\\" + os.path.abspath(directory)

            # abspath() in Python 3.7+ removes trailing path separators (#402)
            if directory[-1] != sep:
                directory += sep

Python docs explain that os.path.abspath() is to convert a relative path to an absolute path. The UNC example is not a relative path, so it is not surprising this fails.

I'm not sure how the path limit should work on Windows, but here's what I'm doing:

        if WINDOWS and len(directory) > 240:
            # Enable longer-than-260-character paths on Windows
            directory = "\\\\?\\" + os.path.abspath(directory)

            # abspath() in Python 3.7+ removes trailing path separators (#402)
            if directory[-1] != sep:
                directory += sep

Since my UNC paths are nowhere near this limit, this works for me!

[AlttiRi]

Well, yes, running gallery-dl in //KEENETIC/USB fails with OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '\\\\'.

Is possible to concat \\?\ verbatim with a network drive?

[AlttiRi]

Possible.

\\?\UNC\

web.archive.org/web/1/docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file

[mfgering]

AlttiRI, thanks for the references. I did more tests and found that "\?\UNC\server\share" works!

Valid, working example:

\\?\UNC\foo.bar\heap\foo\gallery-dl\

But I don't see how to make the current code produce this format.

Currently, if the host is on Windows:

directory = "\\\\?\\" + os.path.abspath(directory)

This option:

directory = "\\\\?\\UNC" + os.path.abspath(directory)

Won't work. The abspath() function wants to add a drive letter to the directory value, which will never work for a shared and unmapped directory.

[AlttiRi]

let directory;
directory = "//KEENETIC/USB";
//directory = "./";

directory = path.resolve(directory);
directory = directory.startsWith("\\\\") ? "\\\\?\\UNC" + directory.slice(1) : "\\\\?\\" + directory;
console.log(directory);
console.log(await fs.readdir(directory));

[mfgering]

Is there a python equivalent (I'm not finding path.resolve() )

[rautamiekka]

Something like

gallery-dl/gallery_dl/cache.py

Line 209 in 0d02a78

cachedir = util.expand_path(os.path.join(cachedir, "gallery-dl"))

, I think.

[mikf]

Should be fixed with the changes in ac80474

The abspath() function wants to add a drive letter to the directory value

Maybe in older Python versions. In 3.10 it works as expected with UNC paths

>>> import ntpath
>>> ntpath.abspath("//server/share/foobar")
'\\\\server\\share\\foobar'
>>> ntpath.abspath("\\\\?\\UNC\\server\\share\\foobar")
'\\\\?\\UNC\\server\\share\\foobar'
>>> 

[mfgering]

With python 3.10, the changes in ac80474 work.

Thanks!