Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use NaN-boxing on value::InnerValue #4091

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
+1,310 −493
Open

Use NaN-boxing on value::InnerValue #4091

Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
6d3d9db
Start of nan boxing values
hansl Dec 19, 2024
c64417d
Continuing, but this endian thing is not helping
hansl Dec 20, 2024
e357908
Finish the boxing and add tests
hansl Dec 20, 2024
d16a546
Better tests and fixing a couple of bugs
hansl Dec 20, 2024
e2496c3
Rollback changes outside of core/engine/src/value/...
hansl Dec 20, 2024
683444e
Fix Drop::drop and clippies
hansl Dec 20, 2024
9d1ac9f
Move MSRV to 1.83
hansl Dec 20, 2024
167d634
Move MSRV back to 1.82 and impl missing const fn
hansl Dec 21, 2024
2a68855
Fix a few clippies and ignore the one thats wrong
hansl Dec 21, 2024
56d513a
Merge branch 'main' into nan_box
hansl Dec 23, 2024
c7cd8b0
WIP
hansl Dec 23, 2024
52fc103
Duh!
hansl Dec 26, 2024
084b25e
Merge remote-tracking branch 'upstream/main' into nan_box
hansl Dec 26, 2024
345cd69
Oops
hansl Dec 26, 2024
710d8f1
Clippies
hansl Dec 26, 2024
c6d0669
refactor bit-masks and ranges
hansl Dec 29, 2024
7ae9352
Merge remote-tracking branch 'upstream/main' into nan_box
hansl Dec 29, 2024
ad64fd9
NAN is not null_ptr and clippies
hansl Dec 29, 2024
ac171fe
Move all magic numbers and bits to the bits private module
hansl Dec 29, 2024
8789d55
Merge remote-tracking branch 'upstream/main' into nan_box
hansl Dec 30, 2024
ec32abd
Implement both enum-based and nan-boxed JsValue inner behind a flag
hansl Dec 30, 2024
40d5e5c
Clippies
hansl Dec 30, 2024
e647385
Remove debug logging
hansl Jan 1, 2025
69bbf82
Some simple attempts to improve performance with NonNull
hansl Jan 5, 2025
ca9f657
Use top 2 bits for pointer tagging, speeding up as_variant
hansl Jan 5, 2025
e29cedb
Remove the feature flag for NaN boxing now that were closer in perf
hansl Jan 5, 2025
c949461
Add a bits section to explain the scheme
hansl Jan 6, 2025
c21c886
Inline ALL THE THINGS - slight increase in perf
hansl Jan 7, 2025
5d406c7
Merge branch 'main' into nan_box
hansl Jan 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 60 additions & 43 deletions core/engine/src/value/inner/nan_boxed.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,21 +19,14 @@
//! | `False` | `7FF6:0000:0000:0000` | |
//! | `True` | `7FF6:0000:0000:0001` | |
//! | `Integer32` | `7FF7:0000:IIII:IIII` | 32-bits integer. |
//! | `BigInt` | `7FF[8-F]:PPPP:PPPP:PPPP \| 0` | 51-bits pointer. Assumes non-null pointer. |
//! | `Object` | `7FF[8-F]:PPPP:PPPP:PPPP \| 1` | 51-bits pointer. |
//! | `Symbol` | `7FF[8-F]:PPPP:PPPP:PPPP \| 2` | 51-bits pointer. |
//! | `String` | `7FF[8-F]:PPPP:PPPP:PPPP \| 3` | 51-bits pointer. |
//! | `BigInt` | `7FF8:PPPP:PPPP:PPPP` | 49-bits pointer. Assumes non-null pointer. |
//! | `Object` | `7FFA:PPPP:PPPP:PPPP` | 49-bits pointer. |
//! | `Symbol` | `7FFC:PPPP:PPPP:PPPP` | 49-bits pointer. |
//! | `String` | `7FFE:PPPP:PPPP:PPPP` | 49-bits pointer. |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So in the pointer space we only have 1 unused slot left right which is 7FF8

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7FF8 (or b0111_1111_1111_1000) is used by bigint if the pointer is non-null. The 16th MSB isn't used (yet?), so pointers are actually 48 bits (I'll correct the comment).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know if the commit helps clarifying this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes sorry I meant to ask if 7FFF is unused and reserved

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is unused and reserved for now.

//! | `Float64` | Any other values. | |
//!
//! Pointers have the highest bit (in the `NaN` tag) set to 1, so they
//! can represent any value from `0x8000_0000_0000` to `0xFFFF_FFFF_FFFF`.
//! The last 2 bits of the pointer is used to store the type of the value.
//!
//! The pointers are assumed to never be NULL, and as such no clash
//! with regular NAN should happen.
//!
//! This only works on 4-bits aligned values, which is asserted when the
//! `InnerValue` is created.
#![allow(clippy::inline_always)]

use crate::{JsBigInt, JsObject, JsSymbol, JsVariant};
Expand Down Expand Up @@ -102,26 +95,40 @@ mod bits {
/// Pointer starting point in `u64`.
pub(super) const POINTER_START: u64 = 0x7FF8_0000_0000_0000;

/// Pointer starting point in `u64`.
pub(super) const POINTER_END: u64 = 0x7FFF_FFFF_FFFF_FFFF;

/// Pointer types mask in `u64`.
pub(super) const POINTER_MASK: u64 = 0x0007_FFFF_FFFF_FFFC;
pub(super) const POINTER_MASK: u64 = 0x0000_FFFF_FFFF_FFFF;

/// Pointer start point for `BigInt` in `u64`.
pub(super) const POINTER_BIGINT_START: u64 = POINTER_START | POINTER_TYPE_BIGINT;
/// Pointer end point for `BigInt` in `u64`.
pub(super) const POINTER_BIGINT_END: u64 = POINTER_START | POINTER_MASK | POINTER_TYPE_BIGINT;
/// Pointer start point for `JsObject` in `u64`.
pub(super) const POINTER_OBJECT_START: u64 = POINTER_START | POINTER_TYPE_OBJECT;
/// Pointer end point for `JsObject` in `u64`.
pub(super) const POINTER_OBJECT_END: u64 = POINTER_START | POINTER_MASK | POINTER_TYPE_OBJECT;
/// Pointer start point for `JsSymbol` in `u64`.
pub(super) const POINTER_SYMBOL_START: u64 = POINTER_START | POINTER_TYPE_SYMBOL;
/// Pointer end point for `JsSymbol` in `u64`.
pub(super) const POINTER_SYMBOL_END: u64 = POINTER_START | POINTER_MASK | POINTER_TYPE_SYMBOL;
/// Pointer start point for `JsString` in `u64`.
pub(super) const POINTER_STRING_START: u64 = POINTER_START | POINTER_TYPE_STRING;
/// Pointer end point for `JsString` in `u64`.
pub(super) const POINTER_STRING_END: u64 = POINTER_START | POINTER_MASK | POINTER_TYPE_STRING;

/// Pointer mask for the type of the pointer.
pub(super) const POINTER_TYPE_MASK: u64 = 0x0003;
pub(super) const POINTER_TYPE_MASK: u64 = 0x0007_0000_0000_0000;

/// Pointer value for `BigInt`.
pub(super) const BIGINT: u64 = 0x0000;
/// Pointer type value for `BigInt`.
pub(super) const POINTER_TYPE_BIGINT: u64 = 0x0000_0000_0000_0000;

/// Pointer value for `JsObject`.
pub(super) const OBJECT: u64 = 0x0001;
/// Pointer type value for `JsObject`.
pub(super) const POINTER_TYPE_OBJECT: u64 = 0x0004_0000_0000_0000;

/// Pointer value for `JsSymbol`.
pub(super) const SYMBOL: u64 = 0x0002;
/// Pointer type value for `JsSymbol`.
pub(super) const POINTER_TYPE_SYMBOL: u64 = 0x0005_0000_0000_0000;

/// Pointer value for `JsString`.
pub(super) const STRING: u64 = 0x0003;
/// Pointer type value for `JsString`.
pub(super) const POINTER_TYPE_STRING: u64 = 0x0006_0000_0000_0000;

/// NAN value in `u64`.
pub(super) const NAN: u64 = 0x7FF8_0000_0000_0000;
Expand Down Expand Up @@ -168,25 +175,27 @@ mod bits {
#[allow(clippy::verbose_bit_mask)]
pub(super) const fn is_bigint(value: u64) -> bool {
// If `(value & POINTER_MASK)` is zero, then it is NaN.
is_pointer(value) && (value & POINTER_TYPE_MASK == BIGINT) && (value & POINTER_MASK) != 0
is_pointer(value)
&& (value & POINTER_TYPE_MASK == POINTER_TYPE_BIGINT)
&& (value & POINTER_MASK) != 0
}

/// Checks that a value is a valid Object.
#[inline(always)]
pub(super) const fn is_object(value: u64) -> bool {
is_pointer(value) && (value & POINTER_TYPE_MASK == OBJECT)
is_pointer(value) && (value & POINTER_TYPE_MASK == POINTER_TYPE_OBJECT)
}

/// Checks that a value is a valid Symbol.
#[inline(always)]
pub(super) const fn is_symbol(value: u64) -> bool {
is_pointer(value) && (value & POINTER_TYPE_MASK == SYMBOL)
is_pointer(value) && (value & POINTER_TYPE_MASK == POINTER_TYPE_SYMBOL)
}

/// Checks that a value is a valid String.
#[inline(always)]
pub(super) const fn is_string(value: u64) -> bool {
is_pointer(value) && (value & POINTER_TYPE_MASK == STRING)
is_pointer(value) && (value & POINTER_TYPE_MASK == POINTER_TYPE_STRING)
}

/// Returns a tagged u64 of a 64-bits float.
Expand Down Expand Up @@ -245,7 +254,7 @@ mod bits {
assert_ne!(value_masked, 0, "Pointer is NULL.");

// Simply cast for bits.
POINTER_START | 0 | value_masked
POINTER_BIGINT_START | value_masked
}

/// Returns a tagged u64 of a boxed `[JsObject]`.
Expand All @@ -270,7 +279,7 @@ mod bits {
assert_ne!(value_masked, 0, "Pointer is NULL.");

// Simply cast for bits.
POINTER_START | 1 | value_masked
POINTER_OBJECT_START | value_masked
}

/// Returns a tagged u64 of a boxed `[JsSymbol]`.
Expand All @@ -295,7 +304,7 @@ mod bits {
assert_ne!(value_masked, 0, "Pointer is NULL.");

// Simply cast for bits.
POINTER_START | 2 | value_masked
POINTER_SYMBOL_START | value_masked
}

/// Returns a tagged u64 of a boxed `[JsString]`.
Expand All @@ -320,7 +329,7 @@ mod bits {
assert_ne!(value_masked, 0, "Pointer is NULL.");

// Simply cast for bits.
POINTER_START | 3 | value_masked
POINTER_STRING_START | value_masked
}

/// Returns a reference to T from a tagged value.
Expand All @@ -343,8 +352,14 @@ const_assert!(f64_is_nan(f64_from_bits(bits::NULL)));
const_assert!(f64_is_nan(f64_from_bits(bits::FALSE)));
const_assert!(f64_is_nan(f64_from_bits(bits::TRUE)));
const_assert!(f64_is_nan(f64_from_bits(bits::INTEGER32_ZERO)));
const_assert!(f64_is_nan(f64_from_bits(bits::POINTER_START)));
const_assert!(f64_is_nan(f64_from_bits(bits::POINTER_END)));
const_assert!(f64_is_nan(f64_from_bits(bits::POINTER_BIGINT_START)));
const_assert!(f64_is_nan(f64_from_bits(bits::POINTER_BIGINT_END)));
const_assert!(f64_is_nan(f64_from_bits(bits::POINTER_OBJECT_START)));
const_assert!(f64_is_nan(f64_from_bits(bits::POINTER_OBJECT_END)));
const_assert!(f64_is_nan(f64_from_bits(bits::POINTER_SYMBOL_START)));
const_assert!(f64_is_nan(f64_from_bits(bits::POINTER_SYMBOL_END)));
const_assert!(f64_is_nan(f64_from_bits(bits::POINTER_STRING_START)));
const_assert!(f64_is_nan(f64_from_bits(bits::POINTER_STRING_END)));

/// A NaN-boxed `[JsValue]`'s inner.
pub(crate) struct NanBoxedValue(pub u64);
Expand Down Expand Up @@ -624,15 +639,17 @@ impl NanBoxedValue {
JsVariant::Integer32(bits::untag_i32(self.0))
}
bits::NAN => JsVariant::Float64(f64::NAN),
bits::POINTER_START..=bits::POINTER_END => {
let ptr = self.0 & bits::POINTER_MASK;
match self.0 & bits::POINTER_TYPE_MASK {
bits::BIGINT => JsVariant::BigInt(unsafe { &*(ptr as *const _) }),
bits::OBJECT => JsVariant::Object(unsafe { &*(ptr as *const _) }),
bits::SYMBOL => JsVariant::Symbol(unsafe { &*(ptr as *const _) }),
bits::STRING => JsVariant::String(unsafe { &*(ptr as *const _) }),
_ => unreachable!(),
}
bits::POINTER_BIGINT_START..=bits::POINTER_BIGINT_END => {
JsVariant::BigInt(unsafe { bits::untag_pointer(self.0) })
}
bits::POINTER_OBJECT_START..=bits::POINTER_OBJECT_END => {
JsVariant::Object(unsafe { bits::untag_pointer(self.0) })
}
bits::POINTER_SYMBOL_START..=bits::POINTER_SYMBOL_END => {
JsVariant::Symbol(unsafe { bits::untag_pointer(self.0) })
}
bits::POINTER_STRING_START..=bits::POINTER_STRING_END => {
JsVariant::String(unsafe { bits::untag_pointer(self.0) })
}
_ => JsVariant::Float64(f64_from_bits(self.0)),
}
Expand Down
Loading