The Rust stdlib can't target osx <=10.6 due to TLS issues

[Manishearth]

See https://bugzilla.mozilla.org/show_bug.cgi?id=1164109

It seems like libstd needs some TLS features introduced in OSX 10.7, and cannot target lower versions of OSX.

While 10.6 lost support last year, we probably should still make it possible for vanilla Rust (not #[no_std]) to target earlier OSX versions, especially so that Rust-in-Gecko doesn't constrain Firefox target platforms. rustc foo.rs should work for a foo.rs that doesn't use TLS.

Perhaps we should stub out TLS on those versions (replace it with a panic), or use a different, perhaps less performant, TLS implementation?

[rillian]

Note that this blocks shipping rust code in Firefox.

[larsbergstrom]

cc @alexcrichton @brson @nikomatsakis

[alexcrichton]

I've discussed this with @brson a bit and the conclusion was that the best way forward is likely to:

• Don't change the standard library by default, continue saying that Rust requires OSX 10.7+
• Add a ./configure option to building Rust itself to enable 10.6 support
• When building the standard library with this ./configure option the TLS support of the standard library will use pthread-based TLS, fixing this incompatibility.

[richo]

Is that going to create a testing nightmare? Does firefox have a slated EOL
date for 10.6 support?

(Sorry to lurk in)

On Wed, May 13, 2015 at 12:59 PM, Alex Crichton [email protected]
wrote:

I've discussed this with @brson https://github.com/brson a bit and the
conclusion was that the best way forward is likely to:

• Don't change the standard library by default, continue saying that
  Rust requires OSX 10.7+
• Add a ./configure option to building Rust itself to enable 10.6
  support
• When building the standard library with this ./configure option the
  TLS support of the standard library will use pthread-based TLS, fixing this
  incompatibility.

—
Reply to this email directly or view it on GitHub
#25342 (comment).

[alexcrichton]

@richo Rust will continue to not officially support OSX 10.6, so we will not be running automated tests. We will, however, accept patches for compatibility.

[larsbergstrom]

@alexcrichton Are you sure you want that? I was sort of hoping that, at least for as long as we can, we would try to use the actual Rust stable released binaries in Gecko, until we start pulling in libraries from Servo that require unstable features or want to cross-compile.

I'm certainly not against having Gecko, say, use the same Rust compiler snapshots that we use in Servo, but didn't want to go down that road unless we had to, to discourage people from writing code that uses nightly/unstable features.

[alexcrichton]

@larsbergstrom it's definitely a bit nuanced, but there are a number of reasons I believe Gecko wants its own build of Rust right now (at least for building releases):

• Gecko wants to disable jemalloc in Rust (it's enabled on all distributions by default right now)
• Gecko probably wants to disable segmented stacks in the standard library (it's not using them at all)
• Gecko probably wants to disable unwinding in the standard library (e.g. trigger an abort)

All of these are planned future extensions to the Rust distribution itself, but not all of these are implemented today.

Specifically on the topic of TLS, however, this isn't a feature that we can do runtime detection and have a "fast path" and a "slow path" both built in. Using the "fast path" of TLS is a compile-time decision, so when you compile the standard library you have to declare whether you want it to target 10.6 or 10.7+. We currently do not have a great way of managing this sort of versioned compilation (hence the hard requirement for 10.7+).

I think it can also work in Gecko to only require a custom build of Rust for the distribution bots instead of all developers. For example I doubt anyone's still developing Gecko seriously on OSX 10.6, so the stable releases should work just fine for that purpose. In that sense all developers of Gecko can just use a normal stable release (e.g. installed on the system), and the release process would just have to be tweaked a bit.

[larsbergstrom]

@alexcrichton Sounds reasonable to me. I just didn't want to go down the private-Rust-builds path without making sure the team was cool with that.

[alexcrichton]

@larsbergstrom yeah makes sense, I can definitely see where it's a huge pain to have to deal with. In terms of a quick solution having a ./configure option for "let's target OSX 10.6" is probably the best, and longer-term solutions may, well, take longer, and also be more complicated. I would prefer a proper solution, for example, to the other problems (below), but they'll take time:

• Remove segmented stacks in favor of stack probes, much less overhead and also no runtime support needed
• Remove libstd's hardwired dependency on jemalloc, instead have the compiler inject it as late as possible and otherwise just bind the malloc/free functions in the standard library. This means that you won't have to recompile Rust to get it to target something other than jemalloc.
• Teach Gecko to LTO for releases which will allow Gecko to strip all unwinding in a release build, removing the code bloat.

At this time I unfortunately don't know of a way to compile the standard library for 10.7+ and retroactively go back and rewrite the TLS accesses to use a 10.6 compatible scheme, but that's not to say one won't come to mind!

[rillian]

@alexcrichton, isn't this just pushing the pain back onto gecko? Making every MacOS X developer add --enable-macos-target=10.7 to their gecko build is better than having to install a custom rust build, but still pretty painful. And then you're further from the configuration the integration servers use, which is currently pretty close on Mac.

Having an explicit configure switch for rustc would at least make it more obvious what's going on.

At this time I unfortunately don't know of a way to compile the standard library for 10.7+ and retroactively go back and rewrite the TLS accesses to use a 10.6 compatible scheme, but that's not to say one won't come to mind!

Is there an llvm optimizer we could run to strip out the thread local functions if they're not used by the input code? Or is the unwind stuff you mentioned what's keeping it alive in the first place?

[alexcrichton]

@rillian

isn't this just pushing the pain back onto gecko?

Unfortunately yeah this would be pushing some logic down onto Gecko's build system. I figured it wouldn't be too hard to pass the 10.7 argument on most OSX builds (except the integration servers), but I can see where having the two be configured closely is nice!

Having an explicit configure switch for rustc would at least make it more obvious what's going on.

Unfortunately this would entail shipping an entirely separate copy of the standard library by default, which is a pretty big step to take for this minor issue.

Is there an llvm optimizer we could run to strip out the thread local functions if they're not used by the input code?

We could in theory write a pass such as this perhaps (this does not exist in any form that I know of), but it would require that all builds on OSX for rust be compiled with LTO (to be able to retroactively modify code) which can be quite slow, especially for development.

Or is the unwind stuff you mentioned what's keeping it alive in the first place?

Nah the unwind stuff is orthogonal to the TLS stuff, there's a pretty hard dependency on OS-based TLS which we can't sever.

[rillian]

Another angle is to improve rustc's support for external runtime linkage. If we could use --emit obj and have gecko's build system link to a separate build of libstd without tls, that would be lighter weight than needing a special toolchain.

Is that more promising? I tried that approach first, but couldn't find a well-defined reference the runtime, and link always died looking for llvm's __morestack, so I switched to using the staticlib output.

[alexcrichton]

@rillian my comment here may explain why using --emit obj is somewhat difficult, but there's also the problem of ABI instability. The compiler currently cannot link to a standard library that was produced by another compiler revision due to the ABI changing over time, so you'd have to mandate that all Rust compilers are the same revision as the one which built libstd without TLS.

[rillian]

Thanks @alexcrichton, I hadn't seen your reply to the gecko --emit=obj bug.

Matching ABI doesn't introduce a different requirement from needing a pthread-tls toolchain, but I guess the failure mode is different. Is there something to always reject linking mis-matched ABI objects, or is it like C++ ABI skew where one only finds some problems at runtime?

[rillian]

I've moved gecko tracking of this issue to https://bugzilla.mozilla.org/show_bug.cgi?id=1166996

In the interest of making progress here, can we request a configuration switch for rustc to enable support for 10.6 from the rust team? I know how to add configure switches but not how to stub out or replace the std::thread functions. I can then build and upload a custom toolchain to use on mac, which would let us ship.

[glandium]

Xcode toolchains (gcc in the past, and now clang) allow developers to change what minimum version of OSX the build binary is going to support by setting the MACOSX_DEPLOYMENT_TARGET. It would make sense for the rust compiler to do the same, and switch TLS implementations depending on its value or lack thereof. There also is a -mmacosx-version-min build flag for gcc/clang with the same meaning as the MACOSX_DEPLOYMENT_TARGET environment variable.

[alexcrichton]

@glandium oh sorry for not responding sooner, but thanks for the info! Perhaps the compiler could detect MACOSX_DEPLOYMENT_TARGET and set some form of #[cfg] based on this, but it would unfortunately still mean that the standard library itself needs to be either always compiled for 10.6 (e.g. using slower TLS) or it must be recompiled to target 10.6. I think for now so long as Gecko is OK using a custom build of Rust that we'll stick to "officially supporting" 10.7+, but if it becomes too much of a pain we can possibly reconsider!

[rillian]

For parity with clang and gcc, the rustc build would need to make a version of libstd with both values of the #[cfg] and choose between them at runtime. That's the best suggestion I've heard so far, but the custom toolchain build option will at least unblock gecko.

[alexcrichton]

Yeah if we wanted to officially support 10.6 then I think we'd do something like:

• Always compile the standard library targeting 10.6 for OSX
• Detect MACOSX_DEPLOYMENT_TARGET and provide an appropriate #[cfg] from the compiler
• Use this #[cfg] when defining a thread local variable to select between fast TLS and pthread TLS at compile-time.

Right now I think we don't want to do the first bullet point, but times may change!