"Cannot import files outside of module root directory"

[juliangruber]

WORKAROUND TO FIX THE ISSUE MANUALLY

1. Quit Station
2. Remove the whole spark folder (AppData\Local\Temp\Filecoin Station Desktop\sources\spark)
3. Start Station again

---

This error was submitted by a user in https://filecoinproject.slack.com/archives/C03S6LXSRB8/p1707311859950409:

[2024-02-07T13:25:54Z ERROR zinniad] Cannot import files outside of module root directory \\?\C:\Users\Administrator\AppData\Local\Temp\Filecoin Station Desktop\sources\voyager. file:///C:/Users/ADMINI~1/AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/voyager/main.js

{"type":"activity:info","module":"Zinnia","message":"Voyager exited with code: 1"}

This is the first time I've seen that error, other installations are looking fine.

Some suspicions:

• Could this be a filesystem case sensitive problem?
• Is this another error masking as this one?

Zinnia source of error:

zinnia/runtime/module_loader.rs

Line 99 in ca764ff

format!("Cannot import files outside of module root directory {}. ", module_root.unwrap().display())

Mentioned Voyager file: https://github.com/filecoin-station/voyager/blob/4f4410c36df95774e6f52cabf717c322e27e6693/main.js

[bajtos]

I find it suspicious that module_root starts with \\?. It looks like some kind of a path to a network-shared drive.

[juliangruber]

Could be! But I don't think so necessarily https://stackoverflow.com/questions/21194530/what-does-mean-when-prepended-to-a-file-path

[bajtos]

Could be! But I don't think so necessarily https://stackoverflow.com/questions/21194530/what-does-mean-when-prepended-to-a-file-path

Great find!

Another interesting fact:

The module toot directory contains Administrator among the directory names, while the module URL contains AMINI~1.

I suspect Deno uses a process for converting paths to strings on Windows that is different from Zinnia's.

@juliangruber What is the impact of this issue? How many users are affected?

[bajtos]

I think this may be fixed by filecoin-station/core#345. Let's see if the problem persists after people update their Station versions to apply that patch.

[juliangruber]

yeah! I'm releasing the changes atm, to validate this

[juliangruber]

Released in https://github.com/filecoin-station/desktop/releases/tag/v1.3.8, asked user for confirmation

[juliangruber]

Let's reopen if the issue comes back

[juliangruber]

This issue came back. This time it was also captured with Spark. The paths all look correct. I assume this is a cross platform path handling issue.

[2024-02-21T18:21:47Z INFO zinniad] Starting zinniad with config CliArgs { wallet_address: "", state_root: "C:\Users\\AppData\Local\Filecoin Station Desktop\modules\zinnia", cache_root: "C:\Users\\AppData\Local\Temp\Filecoin Station Desktop\modules\zinnia", files: ["spark\main.js"] }

[2024-02-21T18:21:47Z INFO lassie] Starting Lassie Daemon
[2024-02-21T18:21:47Z INFO lassie] Lassie Daemon is listening on port 64945
{"type":"activity:info","module":"spark","message":"Spark started"}

[2024-02-21T18:21:47Z INFO zinniad] Starting module file:///C:/Users//AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/spark/main.js

[2024-02-21T18:21:48Z ERROR zinniad] Cannot import files outside of module root directory \?\C:\Users<REDACTED>\AppData\Local\Temp\Filecoin Station Desktop\sources\spark. file:///C:/Users//AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/spark/main.js

{"type":"activity:info","module":"Zinnia","message":"Spark exited with code: 1"}
{"type":"activity:error","module":"Zinnia","message":"Zinnia crashed"}
Zinnia closed all stdio with codes 1, 1

I formatted the logs for better readability.

Suspicious: Zinnia is seeing relative paths as args files: ["spark\\main.js"], but then printing absolute paths when starting Starting module file:///C:/Users/<REDACTED>/AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/spark/main.js That's not it. The same output can be seen on macOS, where this works

[juliangruber]

I cannot reproduce this on my Windows 11 Pro laptop

[juliangruber]

I have asked the issue reporter for additional details on their setup. This bug is preventing them from using Zinnia.

[bajtos]

Can you create a user on your Windows laptop with a long username and/or a username containing non-ASCII characters? Would that reproduce the problem?

I assume this is a cross platform path handling issue.

Yeah, I am thinking the same. Windows have several different ways how to encode filesystem paths. Preserving backwards compatibility all the way back to MS-DOS and 8.3 filename format comes with costs.

[juliangruber]

Can you create a user on your Windows laptop with a long username and/or a username containing non-ASCII characters? Would that reproduce the problem?

From the original error reporting I can tell that the username in question consists of [a-z]{5}, so that seems not to be it.

Yeah, I am thinking the same. Windows have several different ways how to encode filesystem paths. Preserving backwards compatibility all the way back to MS-DOS and 8.3 filename format comes with costs.

😱

[bajtos]

@juliangruber what's the severity/priority of this issue? Should we prioritise it for the next milestone?

[juliangruber]

We have only heard about this from one user. They are not able to participate at all because of this. We don't know how many blocked users we aren't aware of. I propose we as a team discuss the severity of this issue.

[bajtos]

It happened again, see filecoin-station/desktop#1639

[2024-06-28T04:24:00Z INFO  zinniad] Starting module file:///C:/Users/Win/AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/spark/main.js
[2024-06-28T04:24:00Z ERROR zinniad] Cannot import files outside of module root directory \\?\C:\Users\Win\AppData\Local\Temp\Filecoin Station Desktop\sources\spark. file:///C:/Users/Win/AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/spark/main.js

[bajtos]

@teo4RM4Z @nekros05 Do you have any idea why Station sees the path where it's installed with the prefix \?\C:\Users?

Edit: I think the prefix is actually \\?\C:\Users

[nekros05]

I don't know, it was installed in the default routes, the real route is this "C:\Users", could you tell me where that route is saved? Maybe I can change it and it can be fixed

[teo4RM4Z]

@teo4RM4Z @nekros05您是否知道为什么 Station 会看到带有前缀的安装路径\?\C:\Users？

编辑：我认为前缀实际上是\\?\C:\Users

I I have uninstalled Filecoin station
Redownloaded 1.8.6 and install to C:\filecoin station
The Crashed still exists .

New log:
station-modules-1719740157976.log

[bajtos]

Zinniad logs from the log file attached in the comment above:

Starting module file:///C:/Users/testname/AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/spark/main.js
Cannot import files outside of module root directory \\?\C:\Users\testname\AppData\Local\Temp\Filecoin Station Desktop\sources\spark. file:///C:/Users/testname/AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/spark/main.js

Additional material I found on the internet:

https://fleexlab.blogspot.com/2016/07/what-are-those-backslash-double.html

(...) \\?\ paths, which are literal extended-length paths that you can use to access or create file system objects with weird characters in the name.

https://users.rust-lang.org/t/understanding-windows-paths/58583/2

The term "verbatim path" is used to refer to the \\?\ prefixes which allow for extended-length paths.
(...)
If you don't use \\?\ then Win32 APIs do a series of transformations to work out which path you meant.

https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file

[bajtos]

@nekros05 @teo4RM4Z

I improved the log messages for this error to include additional information, I hope it will help us to understand better the problem and how to fix it.

Can you please upgrade your Station Desktop to the latest versions (1.9.0) and send us the new error messages from the module log file?

[nekros05]

@bajtos

New log ver 1.9.0

station-modules-1720104832849.log

[bajtos]

Thank you, @nekros05.

The logs confirmed my suspicion. The Windows path created from the URL of the imported module starts with C:\Users while the Windows path obtained via Rust API starts with \\?\C:\Windows.

[bajtos]

Could you please install v2.0.1 and upload the new module logs?

I reworked the module sandboxing in Zinnia. Either the problem will go away, or else I should hopefully get enough the information to fix it.

[nekros05]

@bajtos same error. New log:

station-modules-1720479195227.log

[bajtos]

Cross-posting the relevant part from the module log:

Cannot canonicalize module path: El sistema no puede encontrar el archivo especificado. (os error 2).
Module URL: file:///C:/Users/Win/AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/spark/main.js

@nekros05 can you please try the following steps?

1. Stop Filecoin Station Desktop
2. Delete everything in C:\Users\Win\AppData\Local\Temp\Filecoin Station Desktop
3. Start Filecoin Station Desktop

@juliangruber It looks like the main module file (C:\Users\Win\AppData\Local\Temp\Filecoin Station Desktop\sources\spark\main.js) does not exist. When we call canonicalize, it fails with the error 2:

https://learn.microsoft.com/en-us/windows/win32/debug/system-error-codes--0-499-

ERROR_FILE_NOT_FOUND
2 (0x2)
The system cannot find the file specified.

Do you have any idea why that could happen?

[bajtos]

Related Slack chat thread: https://space-meridian.slack.com/archives/C03S6LXSRB8/p1720474299839619?thread_ts=1720324428.951409&cid=C03S6LXSRB8

[nekros05]

Hi @bajtos , it is already working, but now there is a new problem, the window goes blank after 2-3 seconds of opening it. Version 2.0.2

[bajtos]

Hi @bajtos , it is already working, but now there is a new problem: the window goes blank after 2-3 seconds of opening it. Version 2.0.2

Please open a new GitHub issue for that in the Station Desktop repo:

https://github.com/filecoin-station/desktop/issues/new

[bajtos]

The issue is still happening, see https://space-meridian.sentry.io/issues/5258168696/
There is no \\? in the path in this case.

Zinnia main loop errored ExecaError: Command failed with exit code 1: "C:\Users\ETCPOW\AppData\Local\Programs\Filecoin Station\resources\core\modules\zinnia\zinniad.exe" "spark\main.js"

[2024-09-16T12:40:17Z INFO  zinniad] Starting zinniad with config CliArgs { wallet_address: "0xF23d36AF30191346DB8e141196Ee278A883bfd3B", station_id: "302a300506032b65700321001f779dd3f98339e2cb55fdd91073cfd01501da448a10581008944999e7cd13ab", state_root: "C:\\Users\\ETCPOW\\AppData\\Local\\Filecoin Station Desktop\\modules\\zinnia", cache_root: "C:\\Users\\ETCPOW\\AppData\\Local\\Temp\\Filecoin Station Desktop\\modules\\zinnia", files: ["spark\\main.js"] }
[2024-09-16T12:40:17Z INFO  lassie] Starting Lassie Daemon
[2024-09-16T12:40:17Z INFO  lassie] Lassie Daemon is listening on port 49957
[2024-09-16T12:40:17Z INFO  zinniad] Starting module file:///C:/Users/ETCPOW/AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/spark/main.js
[2024-09-16T12:40:17Z ERROR zinniad] Cannot canonicalize module path: The system cannot find the file specified. (os error 2).
    Module file path: C:\Users\ETCPOW\AppData\Local\Temp\Filecoin Station Desktop\sources\spark\main.js
    Module URL: file:///C:/Users/ETCPOW/AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/spark/main.js

    at getFinalError (file:///C:/Users/ETCPOW/AppData/Local/Programs/Filecoin%20Station/resources/core/node_modules/execa/lib/return/final-error.js:6:9)
    at makeError (file:///C:/Users/ETCPOW/AppData/Local/Programs/Filecoin%20Station/resources/core/node_modules/execa/lib/return/result.js:108:16)
    at getAsyncResult (file:///C:/Users/ETCPOW/AppData/Local/Programs/Filecoin%20Station/resources/core/node_modules/execa/lib/methods/main-async.js:167:4)
    at handlePromise (file:///C:/Users/ETCPOW/AppData/Local/Programs/Filecoin%20Station/resources/core/node_modules/execa/lib/methods/main-async.js:150:17)
    at async file:///C:/Users/ETCPOW/AppData/Local/Programs/Filecoin%20Station/resources/core/lib/zinnia.js:165:9
    at async catchChildProcessExit (file:///C:/Users/ETCPOW/AppData/Local/Programs/Filecoin%20Station/resources/core/lib/zinnia.js:176:5)
    at async Promise.all (index 1)
    at async run (file:///C:/Users/ETCPOW/AppData/Local/Programs/Filecoin%20Station/resources/core/lib/zinnia.js:310:5)
    at async Promise.all (index 0)
    at async station (file:///C:/Users/ETCPOW/AppData/Local/Programs/Filecoin%20Station/resources/core/commands/station.js:113:3) 

[sentry-io]

Sentry Issue: STATION-CORE-AK4

[bajtos]

https://discord.com/channels/1267475749615698061/1267506556216414293/1291127136358502470

[2024-10-02T19:55:17Z INFO  zinniad] Starting module file:///C:/Users/maxvo/AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/spark/main.js
{"type":"activity:info","module":"spark","message":"Spark started"}
[2024-10-02T19:55:17Z ERROR zinniad] Cannot canonicalize module path: The system cannot find the file specified. (os error 2).
    Module file path: C:\Users\maxvo\AppData\Local\Temp\Filecoin Station Desktop\sources\spark\main.js
    Module URL: file:///C:/Users/maxvo/AppData/Local/Temp/Filecoin%20Station%20Desktop/sources/spark/main.js

[juliangruber]

I have a suspicion that this error happens when the file to be imported can't be found. Can we stat it first?

[bajtos]

I have a suspicion that this error happens when the file to be imported can't be found.

Yeah, that seems to be what the error message is telling us.

Can we stat it first?

I propose a different change since this operation usually succeeds + learning that the file does not exist does not help us that much:

When we cannot canonicalize the module path, we can walk the directory tree from the file up to the root until we find a directory that exists. Then, we print the full path of that directory.

WDYT?

[juliangruber]

Sgtm to dig deeper into this error only once it happened.

I would still like to stat it first, so that we have certainty that that is the issue (and not eg that it's not readable or there was a read error etc).

Afterwards good idea to find the next parent path that exists, and report that.

[zipiju]

Had probably something similar happen to me today after restarting the machine.
It looks like the main.js file was actually missing in the /sources/spark directory:

Tried reinstalling the station, but that didn't help.
Then I shut the station down, removed the whole spark folder and started the station again, which recreated the files and fixed the issue:

[bajtos]

@zipiju thank you for the comment, it's helpful!

It's great to have a workaround we can offer users to fix the problem manually.

I'll add it to the issue description.

[bajtos]

Cross-posting from the Slack discussion in https://space-meridian.slack.com/archives/C03S6LXSRB8/p1728493053534679

Not sure if it's relevant, but calling out as a detail: Station's install is not on my boot drive.

The user sent me a ZIP archive with files in Station's CACHE_DIR. There is a nested folder structure, but all folders are empty.

[bajtos]

@juliangruber could this be a bug in the tar package we use to unpack the tarball with source files?

https://github.com/filecoin-station/core/blob/ff2a78c42fb3ba3eeae564772297b62388ee955f/lib/modules.js#L214-L221

    if (entry.type === 'file' || entry.type === 'raw') {
      await mkdir(outDir, { recursive: true })
      // `{ strip: 1 }` tells tar to remove the top-level directory (e.g. `mod-peer-checker-v1.0.0`)
      await pipeline(
        /** @type {any} */(entry.content()),
        /** @type {any} */(tar.x({ strip: 1, C: outDir }))
      )
    }

[juliangruber]

It could be! It could also be an issue with w3s. The tar module has 28 million weekly downloads, so it should be quite safe.

I wonder if we can use a simpler approach. We don't need compression, but we do need the bundling into a single tar file - unless we individually download a list of files instead.

We could also add more debugging 🤔

Currently the .x/.extract method takes care of parsing entries, creating directories and writing files. I would like to know more about what it's doing, so that after the tar operation we can try to stat some files and then report an error with helpful context to Sentry.

Looking at the tar api docs https://www.npmjs.com/package/tar:

• Pass { strict: true }

Treat warnings as crash-worthy errors. Default false.

While this would help with visibility, it will also let it crash on recoverable errors, that could be wine.

• Pass { onwarn: fn }

A function that will get called with (code, message, data) for any warnings encountered

Push these onto an array, submit to Sentry

• Pass { onReadEntry: fn }

A function that gets called with (entry) for each entry that passes the filter.

Push these onto an array, submit to Sentry in case files are missing

[juliangruber]

filecoin-station/core#610

[bajtos]

I am closing this issue as fixed. If the problem resurfaces again in the future, let's open a new issue in https://github.com/filecoin-station/core

[pmusser]

Hi all, I'm the one who mentioned the install was on a different drive from my boot drive. I missed the note about the manual fix and just had station shut off for the past week or so. Upon a reboot (not the first since the problem arose, I swear!) today, the problem resolved without my intervention. Don't know if the AppData folders were erased after a certain amount of time or some other thing, but I thought I'd mention it for the record.