intermittent failure to build library project with a dependency on a stenciljs library

[christianacca]

Current Behavior

Our CI pipeline has an intermittent failure that occurs when building two dependent libraries in a integrated mono-repo. The following is the failure that occurs:

Here's the command being run (distributed across 5 nx build agents):

yarn nx affected --parallel=1 --base=$(BASE_SHA) -t lint build build-storybook test e2e

Strangely, this occurs only when building most/all of the projects in the mono-repo, for example, when the nx.json file has been modified. But this behaviour it not consistent - sometimes touching the nx.json results in a successful build of all projects. I suspect this is because the distribution of tasks sometimes result in the build of these dependent projects sometimes occurring on the same nx build agent, and sometimes not.

I have checked that the outputs fields in the project.json files for these two libraries accurately reflect the file locations generated by their build.

Summary for the two libraries

• dependency order: ask-agora-insights-angular -> ask-agora-insights
• ask-agora-insights: a senciljs library that:
  • generates angular component proxies - these end up being written as files to a sub-directory of ask-agora-insights-angular library
  • generates the package library artifacts in the dist folder
• ask-agora-insights-angular: an angular application that packages the generated component proxies from ask-agora-insights
• I have confirmed that building ask-agora-insights locally using remote cache, restores the output locations defined by the project.json

The project config for these two libraries

ask-agora-insights

project.json:

{
  "name": "ask-agora-insights",
  "$schema": "../../node_modules/nx/schemas/project-schema.json",
  "sourceRoot": "libs/ask-agora-insights/src",
  "projectType": "library",
  "targets": {
    "build": {
      "executor": "@nxext/stencil:build",
      "outputs": ["{options.outputPath}", "{workspaceRoot}/libs/ask-agora-insights-angular/src/generated"],
      "options": {
        "projectType": "library",
        "tsConfig": "libs/ask-agora-insights/tsconfig.lib.json",
        "configPath": "libs/ask-agora-insights/stencil.config.ts",
        "outputPath": "dist/libs/ask-agora-insights"
      },
      "configurations": {
        "production": {
          "dev": false,
          "prod": true
        }
      }
    }
  }
}

resolved project config:

ask-agora-insights-angular

project json:

{
  "name": "ask-agora-insights-angular",
  "$schema": "../../node_modules/nx/schemas/project-schema.json",
  "sourceRoot": "libs/ask-agora-insights-angular/src",
  "prefix": "lib",
  "projectType": "library",
  "targets": {
    "build": {
      "executor": "@nx/angular:package",
      "outputs": ["{workspaceRoot}/dist/{projectRoot}"],
      "options": {
        "project": "libs/ask-agora-insights-angular/ng-package.json",
        "tsConfig": "libs/ask-agora-insights-angular/tsconfig.lib.json"
      }
    },
    "lint": {
      "dependsOn": [
        {
          "projects": "ask-agora-insights",
          "target": "build"
        }
      ]
    },
    "build-storybook": {
      "executor": "@storybook/angular:build-storybook",
      "outputs": ["{options.outputDir}"],
      "dependsOn": [
        {
          "projects": "ask-agora-insights",
          "target": "build"
        }
      ],
      "options": {
        "outputDir": "dist/storybook/ask-agora-insights-angular",
        "configDir": "libs/ask-agora-insights-angular/.storybook",
        "browserTarget": "ask-agora-insights-angular:build-storybook",
        "compodoc": false
      },
      "configurations": {
        "ci": {
          "quiet": true
        }
      }
    }
  }
}

resolved project config:

Expected Behavior

To be able to build dependent libraries, regardless of whether the build for each occurs on the same or different nx agent ie that the caching and replay of the library output should be consistent and reliable.

GitHub Repo

No response

Steps to Reproduce

Given the inconsistent nature of this problem, building a reproducible repo will involve many apps, each having e2e tests whose duration is between 3-10 minutes each, many libraries, with inter-dependencies. And still this may or may not reproduce the error.

Happy to provide access to the nx cloud logs (CIPE: 409256) as required. Or to try and reproduce the error again, with any additionally telemetry / logging switches enabled.

Nx Report

Node           : 22.6.0
OS             : darwin-x64
Native Target  : x86_64-macos
yarn           : 1.22.19

nx                 : 20.0.8
@nx/js             : 20.0.8
@nx/jest           : 20.0.8
@nx/eslint         : 20.0.8
@nx/workspace      : 20.0.8
@nx/angular        : 20.0.8
@nx/cypress        : 20.0.8
@nx/devkit         : 20.0.8
@nx/eslint-plugin  : 20.0.8
@nx/plugin         : 20.0.8
@nx/react          : 20.0.8
@nx/storybook      : 20.0.8
@nx/web            : 20.0.8
@nx/webpack        : 20.0.8
typescript         : 5.5.4
---------------------------------------
Registered Plugins:
@nx/cypress/plugin
@nx/eslint/plugin
@nx/jest/plugin
---------------------------------------
Community plugins:
@ngrx/data                           : 18.1.0
@ngrx/effects                        : 18.1.0
@ngrx/entity                         : 18.1.0
@ngrx/operators                      : 18.1.1
@ngrx/router-store                   : 18.1.0
@ngrx/schematics                     : 18.1.0
@ngrx/store                          : 18.1.0
@ngrx/store-devtools                 : 18.1.0
@nxext/stencil                       : 20.0.6
@progress/kendo-angular-buttons      : 16.11.0
@progress/kendo-angular-dateinputs   : 16.11.0
@progress/kendo-angular-dialog       : 16.11.0
@progress/kendo-angular-dropdowns    : 16.11.0
@progress/kendo-angular-excel-export : 16.11.0
@progress/kendo-angular-grid         : 16.11.0
@progress/kendo-angular-icons        : 16.11.0
@progress/kendo-angular-inputs       : 16.11.0
@progress/kendo-angular-intl         : 16.11.0
@progress/kendo-angular-l10n         : 16.11.0
@progress/kendo-angular-label        : 16.11.0
@progress/kendo-angular-layout       : 16.11.0
@progress/kendo-angular-menu         : 16.11.0
@progress/kendo-angular-navigation   : 16.11.0
@progress/kendo-angular-notification : 16.11.0
@progress/kendo-angular-pdf-export   : 16.11.0
@progress/kendo-angular-popup        : 16.11.0
@progress/kendo-angular-progressbar  : 16.11.0
@progress/kendo-angular-sortable     : 16.11.0
@progress/kendo-angular-tooltip      : 16.11.0
@progress/kendo-angular-treeview     : 16.11.0
@progress/kendo-angular-upload       : 16.11.0
@progress/kendo-angular-utils        : 16.11.0
@rx-angular/cdk                      : 18.0.0
@rx-angular/state                    : 18.1.0
@storybook/angular                   : 8.4.1
@testing-library/angular             : 17.3.1
ng-mocks                             : 14.13.1

Failure Logs

nx run ask-agora-insights-angular:build

It looks like all of ask-agora-insights-angular's dependencies have not been built yet:
- ask-agora-insights

You might be missing a "targetDefaults" configuration in your root nx.json (https://nx.dev/reference/project-configuration#target-defaults),
or "dependsOn" configured in ask-agora-insights-angular's project.json (https://nx.dev/reference/project-configuration#dependson)

---

nx run ask-agora-insights-angular:build-storybook

<s> [webpack.Progress] 5% setup context module factory
<s> [webpack.Progress] 5% setup context module factory
<s> [webpack.Progress] 6% setup before compile
<s> [webpack.Progress] 6% setup before compile angular-compiler
<s> [webpack.Progress] 6% setup before compile ProgressPlugin
<s> [webpack.Progress] 6% setup before compile
<s> [webpack.Progress] 7% setup compile
<s> [webpack.Progress] 7% setup compile ExternalsPlugin
<s> [webpack.Progress] 7% setup compile ExternalsPlugin
<s> [webpack.Progress] 7% setup compile
<s> [webpack.Progress] 8% setup compilation
<s> [webpack.Progress] 8% setup compilation angular-occurrences-plugin
<s> [webpack.Progress] 8% setup compilation angular-compiler
<s> [webpack.Progress] 8% setup compilation mini-css-extract-plugin
<s> [webpack.Progress] 8% setup compilation HtmlWebpackPlugin
<s> [webpack.Progress] 8% setup compilation unplugin-csf
<s> [webpack.Progress] 8% setup compilation storybook-normalize-angular-entry-plugin
<s> [webpack.Progress] 8% setup compilation ArrayPushCallbackChunkFormatPlugin
<s> [webpack.Progress] 8% setup compilation JsonpChunkLoadingPlugin
<s> [webpack.Progress] 8% setup compilation StartupChunkDependenciesPlugin
<s> [webpack.Progress] 8% setup compilation ImportScriptsChunkLoadingPlugin
<s> [webpack.Progress] 8% setup compilation FetchCompileWasmPlugin
<s> [webpack.Progress] 8% setup compilation FetchCompileAsyncWasmPlugin
<s> [webpack.Progress] 8% setup compilation WorkerPlugin
<s> [webpack.Progress] 8% setup compilation SplitChunksPlugin
<s> [webpack.Progress] 8% setup compilation RuntimeChunkPlugin
<s> [webpack.Progress] 8% setup compilation ResolverCachePlugin
<s> [webpack.Progress] 8% setup compilation
<s> [webpack.Progress] 9% setup compilation
<s> [webpack.Progress] 9% setup compilation named-chunks-plugin
<s> [webpack.Progress] 9% setup compilation DedupeModuleResolvePlugin
<s> [webpack.Progress] 9% setup compilation AnyComponentStyleBudgetChecker
<s> [webpack.Progress] 9% setup compilation mini-css-extract-plugin
<s> [webpack.Progress] 9% setup compilation SuppressExtractedTextChunks
<s> [webpack.Progress] 9% setup compilation DefinePlugin
<s> [webpack.Progress] 9% setup compilation ProvidePlugin
<s> [webpack.Progress] 9% setup compilation ProgressPlugin
<s> [webpack.Progress] 9% setup compilation ChunkPrefetchPreloadPlugin
<s> [webpack.Progress] 9% setup compilation SourceMapDevToolPlugin
<s> [webpack.Progress] 9% setup compilation JavascriptModulesPlugin
<s> [webpack.Progress] 9% setup compilation JsonModulesPlugin
<s> [webpack.Progress] 9% setup compilation AssetModulesPlugin
<s> [webpack.Progress] 9% setup compilation DynamicEntryPlugin
<s> [webpack.Progress] 9% setup compilation RuntimePlugin
<s> [webpack.Progress] 9% setup compilation InferAsyncModulesPlugin
<s> [webpack.Progress] 9% setup compilation DataUriPlugin
<s> [webpack.Progress] 9% setup compilation FileUriPlugin
<s> [webpack.Progress] 9% setup compilation CompatibilityPlugin
<s> [webpack.Progress] 9% setup compilation HarmonyModulesPlugin
<s> [webpack.Progress] 9% setup compilation AMDPlugin
<s> [webpack.Progress] 9% setup compilation RequireJsStuffPlugin
<s> [webpack.Progress] 9% setup compilation CommonJsPlugin
<s> [webpack.Progress] 9% setup compilation LoaderPlugin
<s> [webpack.Progress] 9% setup compilation LoaderPlugin
<s> [webpack.Progress] 9% setup compilation NodeStuffPlugin
<s> [webpack.Progress] 9% setup compilation APIPlugin
<s> [webpack.Progress] 9% setup compilation ExportsInfoApiPlugin
<s> [webpack.Progress] 9% setup compilation WebpackIsIncludedPlugin
<s> [webpack.Progress] 9% setup compilation ConstPlugin
<s> [webpack.Progress] 9% setup compilation UseStrictPlugin
<s> [webpack.Progress] 9% setup compilation RequireIncludePlugin
<s> [webpack.Progress] 9% setup compilation RequireEnsurePlugin
<s> [webpack.Progress] 9% setup compilation RequireContextPlugin
<s> [webpack.Progress] 9% setup compilation ImportPlugin
<s> [webpack.Progress] 9% setup compilation ImportMetaContextPlugin
<s> [webpack.Progress] 9% setup compilation SystemPlugin
<s> [webpack.Progress] 9% setup compilation ImportMetaPlugin
<s> [webpack.Progress] 9% setup compilation URLPlugin
<s> [webpack.Progress] 9% setup compilation DefaultStatsFactoryPlugin
<s> [webpack.Progress] 9% setup compilation DefaultStatsPresetPlugin
<s> [webpack.Progress] 9% setup compilation DefaultStatsPrinterPlugin
<s> [webpack.Progress] 9% setup compilation JavascriptMetaInfoPlugin
<s> [webpack.Progress] 9% setup compilation EnsureChunkConditionsPlugin
<s> [webpack.Progress] 9% setup compilation RemoveEmptyChunksPlugin
<s> [webpack.Progress] 9% setup compilation MergeDuplicateChunksPlugin
<s> [webpack.Progress] 9% setup compilation FlagIncludedChunksPlugin
<s> [webpack.Progress] 9% setup compilation SideEffectsFlagPlugin
<s> [webpack.Progress] 9% setup compilation FlagDependencyExportsPlugin
<s> [webpack.Progress] 9% setup compilation FlagDependencyUsagePlugin
<s> [webpack.Progress] 9% setup compilation InnerGraphPlugin
<s> [webpack.Progress] 9% setup compilation MangleExportsPlugin
<s> [webpack.Progress] 9% setup compilation ModuleConcatenationPlugin
<s> [webpack.Progress] 9% setup compilation NoEmitOnErrorsPlugin
<s> [webpack.Progress] 9% setup compilation RealContentHashPlugin
<s> [webpack.Progress] 9% setup compilation WasmFinalizeExportsPlugin
<s> [webpack.Progress] 9% setup compilation NamedModuleIdsPlugin
<s> [webpack.Progress] 9% setup compilation DeterministicChunkIdsPlugin
<s> [webpack.Progress] 9% setup compilation DefinePlugin
<s> [webpack.Progress] 9% setup compilation TerserPlugin
<s> [webpack.Progress] 9% setup compilation TemplatedPathPlugin
<s> [webpack.Progress] 9% setup compilation RecordIdsPlugin
<s> [webpack.Progress] 9% setup compilation WarnCaseSensitiveModulesPlugin
<s> [webpack.Progress] 9% setup compilation IgnoreWarningsPlugin
<s> [webpack.Progress] 9% setup compilation
<s> [webpack.Progress] 10% building
<s> [webpack.Progress] 10% building 0/1 entries 0/0 dependencies 0/0 modules
<s> [webpack.Progress] 10% building import loader ./node_modules/@angular-devkit/build-angular/src/tools/babel/webpack-loader.js
<s> [webpack.Progress] 10% building 0/2 entries 1/2 dependencies 0/1 modules
<s> [webpack.Progress] 10% building 0/2 entries 2/3 dependencies 0/2 modules
<s> [webpack.Progress] 29% building 1/2 entries 6/17 dependencies 1/6 modules
<s> [webpack.Progress] 37% building import loader ./node_modules/@angular-devkit/build-angular/node_modules/@ngtools/webpack/src/ivy/index.js
<s> [webpack.Progress] 37% building 1/2 entries 31/50 dependencies 4/24 modules
<s> [webpack.Progress] 37% building import loader ./node_modules/@storybook/csf-plugin/dist/webpack-loader.js
<s> [webpack.Progress] 37% building import loader ./node_modules/@storybook/builder-webpack5/dist/loaders/export-order-loader.js
<s> [webpack.Progress] 37% building 1/2 entries 37/56 dependencies 7/29 modules
<s> [webpack.Progress] 56% building 2/2 entries 79/94 dependencies 12/48 modules
<s> [webpack.Progress] 99% cache begin idle
<s> [webpack.Progress] 99% cache begin idle
<s> [webpack.Progress] 100% 

<s> [webpack.Progress] 99% cache shutdown
<s> [webpack.Progress] 99% cache shutdown
<s> [webpack.Progress] 100% 

=> Failed to build the preview
SB_BUILDER-WEBPACK5_0002 (WebpackInvocationError): Module not found: Error: Can't resolve '../generated' in './libs/ask-agora-insights-angular/src/stories'
    at ./node_modules/@storybook/builder-webpack5/dist/index.js:1:24769
    at ./node_modules/webpack/lib/HookWebpackError.js:67:2
    at Hook.eval [as callAsync] (eval at create (./node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:11:1)
    at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (./node_modules/tapable/lib/Hook.js:18:14)
    at Cache.shutdown (./node_modules/webpack/lib/Cache.js:154:23)
    at ./node_modules/webpack/lib/Compiler.js:1379:15
    at Hook.eval [as callAsync] (eval at create (./node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
    at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (./node_modules/tapable/lib/Hook.js:18:14)
    at Compiler.close (./node_modules/webpack/lib/Compiler.js:1372:23)
    at ./node_modules/@storybook/builder-webpack5/dist/index.js:1:24754
    at finalCallback (./node_modules/webpack/lib/Compiler.js:500:32)
    at onCompiled (./node_modules/webpack/lib/Compiler.js:514:20)
    at ./node_modules/webpack/lib/Compiler.js:1327:21
    at eval (eval at create (./node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:39:1)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

<s> [webpack.Progress] 57% building 2/2 entries 82/94 dependencies 31/50 modules

attention => Storybook now collects completely anonymous telemetry regarding usage.
This information is used to shape Storybook's roadmap and prioritize features.
You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
https://storybook.js.org/telemetry


 NX   Broken build, fix the error above.

You may need to refresh the browser.

Package Manager Version

No response

Operating System

• macOS
• Linux
• Windows
• Other (Please specify)

Additional Information

No response

[xiongemi]

there is a known bug in version 20.0.8 (#28788), could you update to version >=20.0.11 and try again?