Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Might fix issue 58 #62

Merged
merged 3 commits into from
Feb 26, 2021
Merged

Might fix issue 58 #62

merged 3 commits into from
Feb 26, 2021

Conversation

lemire
Copy link
Collaborator

@lemire lemire commented Feb 26, 2021

Fixes #58

@lemire
Copy link
Collaborator Author

lemire commented Feb 26, 2021

@gfoidl So hopefully you see what I had in mind. It still leaves open bug possibilities with big endian systems because we do not test. We are smart people but even smart people can't predict what could go wrong on systems they do not have access to.

@gfoidl
Copy link
Contributor

gfoidl commented Feb 26, 2021

Yep, that's fine. My mind was too knotty here...

In #49 care needs to be taken at

unsafe internal static uint parse_eight_digits_unrolled(byte* chars)
{
ulong val = Unsafe.ReadUnaligned<ulong>(chars);
return parse_eight_digits_unrolled(val);
}

unsafe internal static bool is_made_of_eight_digits_fast(byte* chars)
{
ulong val = Unsafe.ReadUnaligned<ulong>(chars);
return is_made_of_eight_digits_fast(val);
}

The Unsafe.ReadUnaligned is like a reinterpret_cast<ulong*> from an unsigned char source in C++ terms.

@lemire
Copy link
Collaborator Author

lemire commented Feb 26, 2021

@gfoidl I am aware, but this is guarded... See...

    unsafe internal static bool is_made_of_eight_digits_fast(ulong val)
    {
      // We only enable paths depending on this function on little endian
      // platforms (it happens to be effectively nearly everywhere).
      return BitConverter.IsLittleEndian && (((val & 0xF0F0F0F0F0F0F0F0) |
               (((val + 0x0606060606060606) & 0xF0F0F0F0F0F0F0F0) >> 4)) ==
              0x3333333333333333);
    }

So is_made_of_eight_digits_fast will always return false on big endian systems.

Copy link
Contributor

@gfoidl gfoidl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then it's perfect, simple and clean 👍

@lemire
Copy link
Collaborator Author

lemire commented Feb 26, 2021

Merging with approval from @gfoidl

@lemire lemire merged commit 8f16c16 into master Feb 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unsafe loads should be guarded with BitConverter.IsLittleEndian or avoided
2 participants