Segfault while using lmdb 2.7.x

[LekoArts]

Hi Kris!

For the Gatsby 5.3 release we bumped our usage of lmdb from 2.5.3 to 2.7.1 (gatsbyjs/gatsby#37160). Gatsby also uses Parcel (which we bumped to 2.8.1) which itself uses 2.5.2 (via @parcel/cache#[email protected]).

Shortly after the release we saw users on Mac (we couldn't reproduce on WSL/Ubuntu or Windows) hit an issue with hard segfaults: gatsbyjs/gatsby#37246

Internally we also saw this and told one of our engineers to attach the segfault-handler:

PID 85025 received SIGSEGV for address: 0x200000001
0   segfault-handler.node               0x000000010838d0aa _ZL16segfault_handleriP9__siginfoPv + 298
1   libsystem_platform.dylib            0x00007ff80064bdfd _sigtramp + 29
2   node                                0x00000001036ef413 _ZN2v88internal7Scanner4NextEv + 1123
3   node.abi108.node                    0x000000010844911d _ZN7DbiWrapC2ERKN4Napi12CallbackInfoE + 1373
4   node.abi108.node                    0x000000010844b6d2 _ZZN4Napi10ObjectWrapI7DbiWrapE26ConstructorCallbackWrapperEP10napi_env__P20napi_callback_info__ENKUlvE_clEv + 66
5   node.abi108.node                    0x000000010844b418 _ZN4Napi10ObjectWrapI7DbiWrapE26ConstructorCallbackWrapperEP10napi_env__P20napi_callback_info__ + 72
6   node                                0x0000000102fe0780 _ZN6v8impl12_GLOBAL__N_123FunctionCallbackWrapper6InvokeERKN2v820FunctionCallbackInfoINS2_5ValueEEE + 128
7   node                                0x000000010320dd98 _ZN2v88internal25FunctionCallbackArguments4CallENS0_15CallHandlerInfoE + 216
8   node                                0x000000010320d564 _ZN2v88internal12_GLOBAL__N_119HandleApiCallHelperILb1EEENS0_11MaybeHandleINS0_6ObjectEEEPNS0_7IsolateENS0_6HandleINS0_10HeapObjectEEESA_NS8_INS0_20FunctionTemplateInfoEEENS8_IS4_EENS0_16BuiltinArgumentsE + 564
9   node                                0x000000010320cf5b _ZN2v88internal21Builtin_HandleApiCallEiPmPNS0_7IsolateE + 219
10  node                                0x0000000103b6edf9 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit + 57
11  node                                0x0000000103af088c Builtins_JSBuiltinsConstructStub + 268
[1]    85025 segmentation fault  NODE_OPTIONS="-r ./seg-fault.js" gatsby build --verbose

I also prepared a repository for you to reproduce it: https://github.com/LekoArts/lmdb-gatsby-5.3.0-segfault-repro
I didn't test yet if this has to do with different lmdb versions (gatsby & parcel) or if this would happen if you'd force lmdb via resolutions to 2.7.x also in Parcel.

For now we're reverting the upgrade (gatsbyjs/gatsby#37255) and it works now again.

[kriszyp]

@LekoArts Have you tried using [email protected]? It does have one important fix for MacOS (#203, although this has been manifested in hanging for other users, so not sure).

Does gatsby use/open any of the same databases that parcel uses (wondering if it is an issue with different versions of lmdb connecting to the same database, which I have tried to maintain, but it is... challenging)?

Anyway, I will try to take a look at this repo, thank you for the detailed info.

[LekoArts]

No, haven't tried 2.7.3 yet!

Does gatsby use/open any of the same databases that parcel uses

No, I don't think so. We use Parcel's LMDBCache (https://github.com/gatsbyjs/gatsby/blob/204825250ca302a73e1ce695222ec29b96112782/packages/gatsby/src/utils/parcel/compile-gatsby-files.ts#L111-L112) but that shouldn't interfere at all with our usage. I just had difficulties with multiple versions for other projects (e.g. graphql, sharp) so it was just an assumption that maybe diff lmdb versions would cause a problem. But then again, 2.5.2 and 2.5.3 seemed to have worked fine together. So maybe that's just a red herring and actual problem is something in > 2.5.3

[janekolszak]

I've seen a segfault in #205 maybe it's connected?

[Relequestual]

@LekoArts Did upgrading fix the issue for you?