Skip to content

Commit

Permalink
Make sentinel value configurable
Browse files Browse the repository at this point in the history
There are OSs that always return the lowest free value.
The algorithm in `lazy_init` always avoids keys with the
sentinel value.
In affected OSs, this means that each call to `lazy_init`
will always request two keys from the OS and returns/frees
the first one (with sentinel value) immediately afterwards.

By making the sentinel value configurable, affected OSs can
use a different value than zero to prevent this performance
issue.
  • Loading branch information
flba-eb committed Dec 6, 2022
1 parent ed61c13 commit 980065a
Showing 1 changed file with 17 additions and 8 deletions.
25 changes: 17 additions & 8 deletions library/std/src/sys_common/thread_local_key.rs
Original file line number Diff line number Diff line change
Expand Up @@ -117,10 +117,14 @@ pub struct Key {
/// This value specifies no destructor by default.
pub const INIT: StaticKey = StaticKey::new(None);

// Define a sentinel value that is unlikely to be returned
// as a TLS key (but it may be returned).
const KEY_SENTVAL: usize = 0;

impl StaticKey {
#[rustc_const_unstable(feature = "thread_local_internals", issue = "none")]
pub const fn new(dtor: Option<unsafe extern "C" fn(*mut u8)>) -> StaticKey {
StaticKey { key: atomic::AtomicUsize::new(0), dtor }
StaticKey { key: atomic::AtomicUsize::new(KEY_SENTVAL), dtor }
}

/// Gets the value associated with this TLS key
Expand All @@ -144,31 +148,36 @@ impl StaticKey {
#[inline]
unsafe fn key(&self) -> imp::Key {
match self.key.load(Ordering::Relaxed) {
0 => self.lazy_init() as imp::Key,
KEY_SENTVAL => self.lazy_init() as imp::Key,
n => n as imp::Key,
}
}

unsafe fn lazy_init(&self) -> usize {
// POSIX allows the key created here to be 0, but the compare_exchange
// below relies on using 0 as a sentinel value to check who won the
// POSIX allows the key created here to be KEY_SENTVAL, but the compare_exchange
// below relies on using KEY_SENTVAL as a sentinel value to check who won the
// race to set the shared TLS key. As far as I know, there is no
// guaranteed value that cannot be returned as a posix_key_create key,
// so there is no value we can initialize the inner key with to
// prove that it has not yet been set. As such, we'll continue using a
// value of 0, but with some gyrations to make sure we have a non-0
// value of KEY_SENTVAL, but with some gyrations to make sure we have a non-KEY_SENTVAL
// value returned from the creation routine.
// FIXME: this is clearly a hack, and should be cleaned up.
let key1 = imp::create(self.dtor);
let key = if key1 != 0 {
let key = if key1 as usize != KEY_SENTVAL {
key1
} else {
let key2 = imp::create(self.dtor);
imp::destroy(key1);
key2
};
rtassert!(key != 0);
match self.key.compare_exchange(0, key as usize, Ordering::SeqCst, Ordering::SeqCst) {
rtassert!(key as usize != KEY_SENTVAL);
match self.key.compare_exchange(
KEY_SENTVAL,
key as usize,
Ordering::SeqCst,
Ordering::SeqCst,
) {
// The CAS succeeded, so we've created the actual key
Ok(_) => key as usize,
// If someone beat us to the punch, use their key instead
Expand Down

0 comments on commit 980065a

Please sign in to comment.