feat(core): lock graph creation when running in another process

[AgentEnder]

Current Behavior

Running Nx in multiple processes at the same time with the daemon disabled can cripple a system due to excess memory usage when creating the graph. This is due to plugin workers being started per-parent process when there is no daemon. This change enables a file lock to prevent the simultaneous processing, and read from the cache when the first run completes.

Currently, running nx show projects 30 times in parallel looks something like this:

30 processes exited within 37535ms

Expected Behavior

30 processes exited within 6435ms

Test Script

//@ts-check

const { spawn } = require('child_process');

let alive = new Set();

let start = Date.now();
let iterations = 30;

for (let i = 0; i < iterations; i++) {
  const cp = spawn('npx nx show projects', [], {
    shell: true,
    env: {
      ...process.env,
      NX_DAEMON: 'false',
      NX_VERBOSE_LOGGING: 'true',
    },
  });
  alive.add(i);
  //   cp.stdout.on('data', (data) => {
  //     console.log(`stdout [${i}]: ${data}`);
  //   });
  cp.stderr.on('data', (data) => {
    console.error(`stderr [${i}]: ${data}`);
  });
  cp.on('exit', (code) => {
    console.log(`child process ${i} exited with code ${code}`);
    alive.delete(i);
  });
}

const i = setInterval(() => {
  if (alive.size > 0) {
  } else {
    clearInterval(i);
    console.log(
      `${iterations} processes exited within ${Date.now() - start}ms`
    );
  }
}, 1);

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
nx-dev	✅ Ready (Inspect)	Visit Preview	Jan 24, 2025 4:54pm

[nx-cloud]

View your CI Pipeline Execution ↗ for commit 73bb882.

Command	Status	Duration	Result
nx affected --targets=lint,test,build,e2e,e2e-c...	✅ Succeeded	5m 37s	View ↗
nx run-many -t check-imports check-commit check...	✅ Succeeded	25s	View ↗
nx-cloud record -- nx-cloud conformance:check	✅ Succeeded	1s	View ↗
nx-cloud record -- nx format:check --base=1c8b3...	✅ Succeeded	2s	View ↗
nx-cloud record -- nx sync:check	✅ Succeeded	<1s	View ↗
nx documentation --no-dte	✅ Succeeded	<1s	View ↗

---

☁️ Nx Cloud last updated this comment at 2025-01-24 18:13:11 UTC

[github-actions]

Failed to publish a PR release of this pull request, triggered by @AgentEnder.
See the failed workflow run at: https://github.com/nrwl/nx/actions/runs/12956983810

[github-actions]

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx [email protected] my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-29408-73bb882

Release details	📑
Published version	0.0.0-pr-29408-73bb882
Triggered by	@AgentEnder
Branch	file-lock-graph
Commit	73bb882
Workflow run	12956983810

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

[github-actions]

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.