[clang] Provide an SSE4.2 implementation of identifier token lexer #68962
Conversation
The _mm_cmpistri instruction can be used to quickly parse identifiers.
With this patch activated, clang pre-processes <iostream> 1.8% faster, and
sqlite3.c amalgametion 1.5% faster, based on time measurements and
number of executed instructions as measured by valgrind.
The introduction of an extra helper function in the regular case has no
impact on performance, see
https://llvm-compile-time-tracker.com/compare.php?from=30240e428f0ec7d4a6d1b84f9f807ce12b46cfd1&to=12bcb016cde4579ca7b75397762098c03eb4f264&stat=instructions:u
llvmbot
added
clang
|
@llvm/pr-subscribers-clang Author: None (serge-sans-paille) ChangesThe _mm_cmpistri instruction can be used to quickly parse identifiers. With this patch activated, clang pre-processes <iostream> 1.8% faster, and sqlite3.c amalgametion 1.5% faster, based on time measurements and number of executed instructions as measured by valgrind. The introduction of an extra helper function in the regular case has no impact on performance, see Full diff: https://github.com/llvm/llvm-project/pull/68962.diff 1 Files Affected:
diff --git a/clang/lib/Lex/Lexer.cpp b/clang/lib/Lex/Lexer.cpp
index feed1b9ecd71a8d..f2d9eb3a8af4e3d 100644
--- a/clang/lib/Lex/Lexer.cpp
+++ b/clang/lib/Lex/Lexer.cpp
@@ -47,6 +47,10 @@
#include <tuple>
#include <utility>
+#ifdef __SSE4_2__
+#include <nmmintrin.h>
+#endif
+
using namespace clang;
//===----------------------------------------------------------------------===//
@@ -1847,19 +1851,46 @@ bool Lexer::LexUnicodeIdentifierStart(Token &Result, uint32_t C,
return true;
}
+static const char *fastParseASCIIIdentifier(const char *CurPtr, const char* BufferEnd) {
+#ifdef __SSE4_2__
+ static constexpr char AsciiIdentifierRange[16] = {
+ '_', '_', 'A', 'Z', 'a', 'z', '0', '9',
+ };
+ constexpr ssize_t BytesPerRegister = 16;
+
+ while (LLVM_LIKELY(BufferEnd - CurPtr >= BytesPerRegister)) {
+ __m128i AsciiIdentifierRangeV = _mm_loadu_si128((const __m128i *)AsciiIdentifierRange);
+
+ __m128i Cv = _mm_loadu_si128((const __m128i *)(CurPtr));
+ int Consumed =
+ _mm_cmpistri(AsciiIdentifierRangeV, Cv,
+ _SIDD_LEAST_SIGNIFICANT | _SIDD_CMP_RANGES |
+ _SIDD_UBYTE_OPS | _SIDD_NEGATIVE_POLARITY);
+ CurPtr += Consumed;
+ if (Consumed == BytesPerRegister)
+ continue;
+ return CurPtr;
+ }
+#else
+ (void)BufferEnd;
+#endif
+
+ unsigned char C = *CurPtr;
+ while (isAsciiIdentifierContinue(C))
+ C = *++CurPtr;
+ return CurPtr;
+}
+
bool Lexer::LexIdentifierContinue(Token &Result, const char *CurPtr) {
// Match [_A-Za-z0-9]*, we have already matched an identifier start.
+
while (true) {
- unsigned char C = *CurPtr;
- // Fast path.
- if (isAsciiIdentifierContinue(C)) {
- ++CurPtr;
- continue;
- }
+
+ CurPtr = fastParseASCIIIdentifier(CurPtr, BufferEnd);
unsigned Size;
// Slow path: handle trigraph, unicode codepoints, UCNs.
- C = getCharAndSize(CurPtr, Size);
+ unsigned char C = getCharAndSize(CurPtr, Size);
if (isAsciiIdentifierContinue(C)) {
CurPtr = ConsumeChar(CurPtr, Size, Result);
continue;
|
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
|
@serge-sans-paille I'm still trying to understand whether this change will be tested by CI - or at all. |
|
ok. I obviously tested on my setup, but having consistent testing is important. Thanks for the quick answer! |
I think we're in good shape for testing because we have at least one bot that's set up to compile for cascade lake, which does support SSE 4.2: https://github.com/llvm/llvm-zorg/blob/4bc450165fa9fcef26adca5a80f4d97b9d6babc3/buildbot/osuosl/master/config/builders.py#L827 so I believe that gets us the test coverage we need. Any disagreement? |
The _mm_cmpistri instruction can be used to quickly parse identifiers.
With this patch activated, clang pre-processes 1.8% faster, and sqlite3.c amalgametion 1.5% faster, based on time measurements and number of executed instructions as measured by valgrind.
The introduction of an extra helper function in the regular case has no impact on performance, see