Too much memory usage on build #58902

alexstine · 2024-02-09T20:44:34Z

Description

I spun up my new WSL VM this afternoon and discovered the npm run build script is using an insane amount of memory. I received a Node out of memory heap error and had to increase using this environment variable.

export NODE_OPTIONS=--max_old_space_size=6000

It looks like at peak build, it nearly uses 3GB of memory which is absolutely insane. It fails right at the TypeScript build.

Step-by-step reproduction instructions

Spin up a WSL2 Ubuntu 20.04 VM.
Clone the repo.
Run npm i && npm run build.
See error.

Screenshots, screen recording, code snippet

No response

Environment info

N/A.

Please confirm that you have searched existing issues in the repo.

Yes

Please confirm that you have tested with all plugins deactivated except Gutenberg.

Yes

The text was updated successfully, but these errors were encountered:

t-hamano · 2024-02-10T02:05:06Z

Thanks for the report.

I have been running the Gutenberg project on WSL2 for over two years, but I have never encountered such an error, nor do I remember operating max_old_space_size.

It looks like at peak build, it nearly uses 3GB of memory which is absolutely insane.

How can I test to measure this memory?

alexstine · 2024-02-10T15:27:53Z

@t-hamano If you do the following, you should be able to see memory usage.

Start a screen session with screen -S build.
Run npm run build.
Press Control+A and then D to detach.
Run free -g to output the memory in GB while the build is running. Run the command multiple times.

Part of me wonders if this is only an issue on the initial build now considering cache is likely used on each build after. This might not be reproducible without starting fresh.

Thanks.

t-hamano · 2024-02-11T05:05:36Z

I performed the following steps.

Cloning the gutenberg repository again.
Run npm i.
Start a screen session with screen -S build.
Run npm run build.
Press Control+A and then D to detach.
Run free -g to output the memory in GB while the build is running. Run the command multiple times.

The average value of the cell where the used column and mem row intersect was 3, and increased to a maximum of 5. Is this measured correctly?

	total	used	free	shared	buff/cache	available
Mem:	31	3	24	0	3	28
Swap:	8	0	8

alexstine · 2024-02-11T15:47:49Z

Yes, that looks correct. I don't ever remember our TypeScript taking that much memory before. Are you using NVM to manage Node?

t-hamano · 2024-02-11T17:08:43Z

Are you using NVM to manage Node?

I'm currently using volta.

alexstine · 2024-03-01T17:28:22Z

I'll leave this open in case anyone else has this problem with NVM, but it seems like I might be the only one. So strange. All in all, the Windows filesystem doesn't perform very well with Node so I guess this should come at no shock to anyone. Mac/Linux handles it much better.

simonhammes · 2024-04-03T20:22:52Z

I am using NVM on Debian Trixie and just got an OOM while running npm run dev.

This was the last command:

node ./bin/packages/validate-typescript-version.js && tsc --build && node ./bin/packages/check-build-type-declaration-files.js

Running export NODE_OPTIONS=--max_old_space_size=6000 before npm run dev fixes the issue.

alexstine · 2024-04-03T21:38:49Z

Yes, I am thinking we should either document this or fix the out of memory error.

benridane · 2024-05-26T15:20:49Z

It appears that the memory-consuming area when tsc --build is run is in packages/components.

212:~/wp1/gutenberg/packages/components$ ../../node_modules/.bin/tsc --build --extendedDiagnostics

<--- Last few GCs --->

[185248:0x62aa000]    45794 ms: Scavenge 972.5 (999.2) -> 969.2 (999.2) MB, pooled: 5 MB, 3.27 / 0.00 ms  (average mu = 0.436, current mu = 0.068) allocation failure;
[185248:0x62aa000]    46331 ms: Mark-Compact 972.5 (999.8) -> 969.1 (1008.5) MB, pooled: 0 MB, 528.37 / 0.01 ms  (average mu = 0.333, current mu = 0.208) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0xe18f84 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
 2: 0x12102f0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 3: 0x12105c7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 4: 0x14400d5  [node]
 5: 0x1459949 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 6: 0x142e018 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 7: 0x142ef45 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 8: 0x1407b8e v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
 9: 0x1868ffc v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
10: 0x1f1e576  [node]
中止 (コアダンプ)

caseymilne · 2024-07-19T04:00:07Z

I've also been unable to get past the initial build due to memory errors:

> [email protected] build:packages
> npm run --silent build:package-types && node ./bin/packages/build.js


<--- Last few GCs --->

[8668:00000294C50F0F90]   148165 ms: Mark-Compact (reduce) 2046.0 (2083.5) -> 2045.5 (2084.3) MB, 623.35 / 0.01 ms  (+ 161.3 ms in 35 steps since start of marking, biggest step 11.5 ms, walltime since start of marking 840 ms) (average mu = 0.422, current [8668:00000294C50F0F90]   149325 ms: Mark-Compact (reduce) 2046.4 (2084.3) -> 2045.6 (2084.5) MB, 1153.60 / 0.00 ms  (average mu = 0.226, current mu = 0.006) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 00007FF63891EEDB node::SetCppgcReference+18123
 2: 00007FF638890C34 DSA_meth_get_flags+89940
 3: 00007FF639307A71 v8::Isolate::ReportExternalAllocationLimitReached+65
 4: 00007FF6392F11C8 v8::Function::Experimental_IsNopFunction+1336
 5: 00007FF639152C70 v8::Platform::SystemClockTimeMillis+659328
 6: 00007FF63914FCF8 v8::Platform::SystemClockTimeMillis+647176
 7: 00007FF63916500A v8::Platform::SystemClockTimeMillis+733978
 8: 00007FF639165887 v8::Platform::SystemClockTimeMillis+736151
 9: 00007FF63916E4BE v8::Platform::SystemClockTimeMillis+772046
10: 00007FF639182CFA v8::Platform::SystemClockTimeMillis+856074
11: 00007FF639182FE3 v8::Platform::SystemClockTimeMillis+856819
12: 00007FF638F117FD v8::base::Thread::StartSynchronously+465613
13: 00007FF638F12897 v8::base::Thread::StartSynchronously+469863
14: 00007FF638F11F54 v8::base::Thread::StartSynchronously+467492
15: 00007FF638E3F4D5 v8::CodeEvent::GetFunctionName+160917
16: 00007FF6393B93FE v8::PropertyDescriptor::writable+678094
17: 00007FF639386168 v8::PropertyDescriptor::writable+468536
18: 00007FF5BAC05A11
'tsc failed. Try cleaning up first: `npm run clean:package-types`'; exit 1
Cannot access this declaration file. You may need to run tsc again: C:\Users\accou\OneDrive\Documentos\gutenberg\packages\core-data\build-types\index.d.ts

Is there any work around? The suggested npm run clean:package-types doesn't resolve this.

sirreal · 2024-07-19T07:14:40Z

I suspect this is related to some types that are very costly to process. @mirka has landed some work on it here: #63388

I haven't seen the memory issues myself recently, but a common fix is to increase the amount of memory Node.js can use. That can be handled by an environment variable. On UNIX machines, this would would be an example of how to do that:

NODE_OPTIONS='--max-old-space-size=8000' npm run build:packages

caseymilne · 2024-07-19T09:25:32Z

I suspect this is related to some types that are very costly to process. @mirka has landed some work on it here: #63388

I haven't seen the memory issues myself recently, but a common fix is to increase the amount of memory Node.js can use. That can be handled by an environment variable. On UNIX machines, this would would be an example of how to do that:
NODE_OPTIONS='--max-old-space-size=8000' npm run build:packages

Good suggestion @sirreal I found this similar command (have not tried it yet) node --max-old-space-size=8000 heapsize.js

caseymilne · 2024-07-19T09:39:49Z

Not sure if this helps but when I'm running build it's stopping with an error about C:\Users\accou\OneDrive\Documentos\gutenberg\packages\dataviews\build-types\index.d.ts and I tried a suggested command npx ts-node C:\Users\accou\OneDrive\Documentos\gutenberg\packages\dataviews\build-types\index.d.ts only to find that this \packages\dataviews\ does not have a "build-types" directory in it. Maybe a previous step is supposed to create it? There is a \tsconfig.json which has "declarationDir": "build-types" in it. Not sure if \build-types is supposed to be present at the start of the process or get built in an earlier step.

sirreal · 2024-07-19T10:22:07Z

That directory should be generated during build, but it may not be if an error occurs before.
Will you share the git commit sha you're on and the exact command and error you're seeing? That will help to continue debugging.

caseymilne · 2024-07-19T12:01:29Z

That directory should be generated during build, but it may not be if an error occurs before. Will you share the git commit sha you're on and the exact command and error you're seeing? That will help to continue debugging.

Commit: 154f1aa, command is npm run build, full output below:

> [email protected] build
> npm run build:packages && wp-scripts build


> [email protected] prebuild:packages
> npm run clean:packages && lerna run build


> [email protected] clean:packages
> rimraf "./packages/*/@(build|build-module|build-style)"

lerna notice cli v7.1.4
lerna info versioning independent

    √  @wordpress/block-serialization-spec-parser:build (2s)
    √  @wordpress/babel-preset-default:build (4s)

 ——————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  Lerna (powered by Nx)   Successfully ran target build for 2 projects (5s)


> [email protected] build:packages
> npm run --silent build:package-types && node ./bin/packages/build.js


<--- Last few GCs --->

[3264:000002935424B0F0]   128868 ms: Mark-Compact (reduce) 2045.1 (2083.5) -> 2045.1 (2084.0) MB, 403.79 / 0.01 ms  (+ 281.6 ms in 71 steps since start of marking, biggest step 6.2 ms, walltime since start of marking 791 ms) (average mu = 0.408, current m[3264:000002935424B0F0]   129739 ms: Mark-Compact (reduce) 2046.1 (2084.0) -> 2045.1 (2084.5) MB, 866.64 / 0.00 ms  (average mu = 0.253, current mu = 0.005) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 00007FF670A8EEDB node::SetCppgcReference+18123
 2: 00007FF670A00C34 DSA_meth_get_flags+89940
 3: 00007FF671477A71 v8::Isolate::ReportExternalAllocationLimitReached+65
 4: 00007FF6714611C8 v8::Function::Experimental_IsNopFunction+1336
 5: 00007FF6712C2C70 v8::Platform::SystemClockTimeMillis+659328
 6: 00007FF6712BFCF8 v8::Platform::SystemClockTimeMillis+647176
 7: 00007FF6712D500A v8::Platform::SystemClockTimeMillis+733978
 8: 00007FF6712D5887 v8::Platform::SystemClockTimeMillis+736151
 9: 00007FF6712DE4BE v8::Platform::SystemClockTimeMillis+772046
10: 00007FF6712F2CFA v8::Platform::SystemClockTimeMillis+856074
11: 00007FF6712F2FE3 v8::Platform::SystemClockTimeMillis+856819
12: 00007FF6710817FD v8::base::Thread::StartSynchronously+465613
13: 00007FF671082897 v8::base::Thread::StartSynchronously+469863
14: 00007FF671081F54 v8::base::Thread::StartSynchronously+467492
15: 00007FF670FAF4D5 v8::CodeEvent::GetFunctionName+160917
16: 00007FF6114DAAFA
'tsc failed. Try cleaning up first: `npm run clean:package-types`'; exit 1
Cannot access this declaration file. You may need to run tsc again: C:\Users\accou\OneDrive\Documentos\gutenberg\packages\core-data\build-types\index.d.ts
PS C:\Users\accou\OneDrive\Documentos\gutenberg>

sirreal · 2024-07-19T15:20:29Z

That looks like a standard out of memory error. The npx ts-node … command isn't going to help here, what may help is setting the environment variable to increase available memory: NODE_OPTIONS='--max-old-space-size=8000'.

Ideally the build would take less memory, but there's still work to be done there 🙂

caseymilne · 2024-07-19T17:32:55Z

Raising memory didn't work for me, but I've probably only got about 7000 available. Maybe later I'll throw it on a server and see if it can build there.

t-hamano · 2024-07-20T10:08:41Z

Many of the people reporting issues here seem to be using Windows OS. I'd love to know if the memory exhaustion is happening on the host OS or on WSL (Ubuntu).

By default, WSL has 50% of the memory that the host OS has: https://learn.microsoft.com/en-us/windows/wsl/wsl-config#main-wsl-settings

Therefore, I think we need to check the following:

Are you building on the host OS or on WSL?
How much memory is on the host OS?
How much memory is allocated to WSL?

sirreal · 2024-10-07T09:10:00Z

This issue is ongoing, there was another report today: https://wordpress.slack.com/archives/C5UNMSU4R/p1728225107128589

benridane · 2024-10-12T06:50:15Z

I also did a build on a 2G memory VPS and it failed, NODE_OPTIONS='--max-old-space-size=8000' had no effect on 2G.

OS: Ubuntu 22.04.2 LTS
memory : 1.9 GiB
Swap:2.0 GiB

benridane · 2024-10-26T08:23:36Z

The build went well with a combination of swap memory (7GB) and 2GB memory.
It seems that the memory required for the build must be at least 6GB including swap. 4GB failed.

OS: Ubuntu 22.04.2 LTS
memory : 7.1 GiB

`top - 13:52:54 up 19 days, 16:09, 2 users, load average: 4.42, 4.29, 2.57
Tasks: 134 total, 1 running, 133 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.6 us, 8.5 sy, 1.8 ni, 5.4 id, 82.2 wa, 0.0 hi, 1.0 si, 0.5 st
MiB Mem : 97.5/1958.5 [|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
MiB Swap: 99.2/2048.0 [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]

top - 14:16:57 up 19 days, 16:33, 2 users, load average: 5.12, 4.24, 4.11
Tasks: 133 total, 1 running, 132 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 5.9 sy, 7.2 ni, 1.9 id, 84.0 wa, 0.0 hi, 0.6 si, 0.4 st
MiB Mem : 98.1/1958.5 [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
MiB Swap: 53.4/7168.0 [||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
`

alexstine added [Type] Bug An existing feature does not function as intended [Priority] High Used to indicate top priority items that need quick attention labels Feb 9, 2024

gziolo added the [Type] Build Tooling Issues or PRs related to build tooling label Feb 10, 2024

sirreal mentioned this issue Dec 30, 2024

Edit Site: Add tsconfig.json validation for package #67406

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Too much memory usage on build #58902

Too much memory usage on build #58902

alexstine commented Feb 9, 2024

t-hamano commented Feb 10, 2024

alexstine commented Feb 10, 2024

t-hamano commented Feb 11, 2024

alexstine commented Feb 11, 2024

t-hamano commented Feb 11, 2024

alexstine commented Mar 1, 2024

simonhammes commented Apr 3, 2024

alexstine commented Apr 3, 2024

benridane commented May 26, 2024 •

edited

Loading

caseymilne commented Jul 19, 2024

sirreal commented Jul 19, 2024

caseymilne commented Jul 19, 2024

caseymilne commented Jul 19, 2024 •

edited

Loading

sirreal commented Jul 19, 2024

caseymilne commented Jul 19, 2024

sirreal commented Jul 19, 2024

caseymilne commented Jul 19, 2024 •

edited

Loading

t-hamano commented Jul 20, 2024

sirreal commented Oct 7, 2024

benridane commented Oct 12, 2024

benridane commented Oct 26, 2024

Too much memory usage on build #58902

Too much memory usage on build #58902

Comments

alexstine commented Feb 9, 2024

Description

Step-by-step reproduction instructions

Screenshots, screen recording, code snippet

Environment info

Please confirm that you have searched existing issues in the repo.

Please confirm that you have tested with all plugins deactivated except Gutenberg.

t-hamano commented Feb 10, 2024

alexstine commented Feb 10, 2024

t-hamano commented Feb 11, 2024

alexstine commented Feb 11, 2024

t-hamano commented Feb 11, 2024

alexstine commented Mar 1, 2024

simonhammes commented Apr 3, 2024

alexstine commented Apr 3, 2024

benridane commented May 26, 2024 • edited Loading

caseymilne commented Jul 19, 2024

sirreal commented Jul 19, 2024

caseymilne commented Jul 19, 2024

caseymilne commented Jul 19, 2024 • edited Loading

sirreal commented Jul 19, 2024

caseymilne commented Jul 19, 2024

sirreal commented Jul 19, 2024

caseymilne commented Jul 19, 2024 • edited Loading

t-hamano commented Jul 20, 2024

sirreal commented Oct 7, 2024

benridane commented Oct 12, 2024

benridane commented Oct 26, 2024

benridane commented May 26, 2024 •

edited

Loading

caseymilne commented Jul 19, 2024 •

edited

Loading

caseymilne commented Jul 19, 2024 •

edited

Loading