Gracefully handle super-long linker command lines #41190
Comments
|
The trouble is that if rustc is generating linker lines that are too long for exec (or the Windows equiv), then the chances are it's also getting a command line that is similarly long. Therefore rustc also needs to accept Are there cases where rustc will generate a linker line that's not proportional to its own command line (setting aside issues around |
|
BTW, the I believe MacOS's linker adds complications in that it can only be flags, not input files - but it supports a second file containing input filenames (which makes intermingling flags and files awkward), with a slightly different format (one filename per line, no quoting rules). |
|
Yes I think that |
|
Some notes:
|
Well, the
I'm not sure what this means. Do you mean that the arg |
|
no I mean it should not remove the file, so you could look at the args when linking fails. |
|
@jsgf ah yes, that is a good point. |
|
For Windows the limit for a command line is a hard 32,768 wchars. Fortunately for Windows, the way that we flatten the command line for
@nagisa Just open the file on Windows with |
|
I know we're currently looking into this but this is causing so much pain for me right now. I'll try moving my I hope it gets fixed very soon. 😭 Can we perhaps assign some labels and look for potential takers? |
alexcrichton
added
A-linkage
|
For whoever is implementing this for msvc, could you also implement deduplication of library input at the same time? cc #38460 |
Mark-Simulacrum
added
the
C-enhancement
This commit adds logic to the compiler to attempt to handle super long linker invocations by falling back to the `@`-file syntax if the invoked command is too large. Each OS has a limit on how many arguments and how large the arguments can be when spawning a new process, and linkers tend to be one of those programs that can hit the limit! The logic implemented here is to unconditionally attempt to spawn a linker and then if it fails to spawn with an error from the OS that indicates the command line is too big we attempt a fallback. The fallback is roughly the same for all linkers where an argument pointing to a file, prepended with `@`, is passed. This file then contains all the various arguments that we want to pass to the linker. Closes rust-lang#41190
This commit adds logic to the compiler to attempt to handle super long linker invocations by falling back to the `@`-file syntax if the invoked command is too large. Each OS has a limit on how many arguments and how large the arguments can be when spawning a new process, and linkers tend to be one of those programs that can hit the limit! The logic implemented here is to unconditionally attempt to spawn a linker and then if it fails to spawn with an error from the OS that indicates the command line is too big we attempt a fallback. The fallback is roughly the same for all linkers where an argument pointing to a file, prepended with `@`, is passed. This file then contains all the various arguments that we want to pass to the linker. Closes rust-lang#41190
rustc: Attempt to handle super long linker invocations This commit adds logic to the compiler to attempt to handle super long linker invocations by falling back to the `@`-file syntax if the invoked command is too large. Each OS has a limit on how many arguments and how large the arguments can be when spawning a new process, and linkers tend to be one of those programs that can hit the limit! The logic implemented here is to unconditionally attempt to spawn a linker and then if it fails to spawn with an error from the OS that indicates the command line is too big we attempt a fallback. The fallback is roughly the same for all linkers where an argument pointing to a file, prepended with `@`, is passed. This file then contains all the various arguments that we want to pass to the linker. Closes #41190
The compiler should gracefully handle invocations of the linker which would otherwise blow the system limits. I've heard reports of this happening on both Unix and Windows, so this is a cross-platform concern.
My preference of how to implement this would be:
@filewherefileis a path on the filesystem that contains a bunch of options.@filesyntax for debuggability@filewhen the command line gets too big.Command, likely well below the platform specific limitsI don't know the precise syntax of these files, unfortunately, but I'm sure google does somewhere!
The text was updated successfully, but these errors were encountered: