[wasm] Rationalize wasm jobs run on CI (#65035)

* Rationalize wasm jobs run on CI

Essentially:
- For rolling builds, we want to run all the wasm jobs, unconditionally
   - And `runtime-extra-platforms` runs the tests that were not in `runtime`, and `runtime-staging`
- For PRs, we use the various path checks

Changes:
- split wasm jobs into various templates
- combine, and simplify some jobs

* `runtime-extra-platforms`, and `runtime-wasm` run only when manually
  triggered with a comment - `/azp run <pipeline-name>`
* `runtime`, and `runtime-staging`, run jobs only when relevant paths
  change. And for `EAT`, and `AOT`, only smoke tests are run.
* And when `runtime-wasm` is triggered manually, it runs *all* the wasm
  jobs completely

| .                 | runtime               | runtime-staging         | runtime-extra-platforms(manual only) | runtime-wasm (manual only) |
| ----------------- | --------------------  | ---------------         | ------------------------------------ | -------                    |
| libtests          | linux: all,   only-pc | windows: all,   only-pc | linux+windows: all, only-pc          | linux+windows: all, always |
| libtests eat      | linux: smoke, only-pc | -                       | linux:         all, only-pc          | linux:         all, always |
| libtests aot      | linux: smoke, only-pc | windows: smoke, only-pc | linux+windows: all, only-pc          | linux+windows: all, always |
|                   |                       |                         |                                      |                            |
| Wasm.Build.Tests  | linux:        only-pc | windows:        only-pc | linux+windows: only-pc               | linux+windows              |
| Debugger tests    | -                     | linux+windows:  only-pc | linux+windows: only-pc               | linux+windows              |
| Runtime tests     | linux:        only-pc | -                       | linux: only-pc                       | linux                      |

* `runtime`, and `runtime-staging`, run all the wasm jobs unconditionally, but `EAT`, and `AOT` still run only smoke tests.
* `runtime-extra-platforms` also runs by default. And it runs only the cases not covered by the above two pipelines.

* jobs w/o `only-pc` are always run

| .                 | runtime                   | runtime-staging       | runtime-extra-platforms (always run) | runtime-wasm (manual only) |
| ----------------- | -------------             | ---------------       | ------------------------------------ | ------                     |
| libtests          | linux: all(v8/chr/node)   | windows: all          | linux+windows: all with node         | N/A                        |
| libtests eat      | linux: smoke              | -                     | linux: all                           |                            |
| libtests aot      | linux: smoke              | windows: smoke        | linux+windows: all                   |                            |
|                   |                           |                       |                                      |                            |
| Wasm.Build.Tests  | linux: always             | windows: always       | none                                 |                            |
| Debugger tests    | -                         | linux+windows: always | none                                 |                            |
| Runtime tests     | linux: always             | -                     | none                                 |                            |

* [wasm] Run tests with node only on runtime-wasm

eng/pipelines/common/templates/wasm-build-tests.yml

@@ -0,0 +1,41 @@
+parameters:
+  alwaysRun: false
+  isExtraPlatformsBuild: false
+  platforms: []
+
+jobs:
+
+#
+# Build the whole product using Mono and run libraries tests, for Wasm.Build.Tests
+#
+- template: /eng/pipelines/common/platform-matrix.yml
+  parameters:
+    jobTemplate: /eng/pipelines/common/global-build-job.yml
+    helixQueuesTemplate: /eng/pipelines/libraries/helix-queues-setup.yml
+    buildConfig: Release
+    runtimeFlavor: mono
+    platforms: ${{ parameters.platforms }}
+    variables:
+      # map dependencies variables to local variables
+      - name: wasmbuildtestsContainsChange
+        value: $[ dependencies.evaluate_paths.outputs['SetPathVars_wasmbuildtests.containsChange'] ]
+      - name: alwaysRunVar
+        value: ${{ parameters.alwaysRun }}
+    jobParameters:
+      isExtraPlatforms: ${{ parameters.isExtraPlatformsBuild }}
+      testGroup: innerloop
+      nameSuffix: WasmBuildTests
+      buildArgs: -s mono+libs+host+packs+libs.tests -c $(_BuildConfig) /p:ArchiveTests=true /p:TestWasmBuildTests=true /p:TestAssemblies=false /p:BrowserHost=$(_hostedOs)
+      timeoutInMinutes: 180
+      condition: >-
+        or(
+          eq(variables['alwaysRunVar'], true),
+          eq(dependencies.evaluate_paths.outputs['SetPathVars_wasmbuildtests.containsChange'], true))
+      # extra steps, run tests
+      extraStepsTemplate: /eng/pipelines/libraries/helix.yml
+      extraStepsParameters:
+        creator: dotnet-bot
+        testRunNamePrefixSuffix: Mono_$(_BuildConfig)_$(_hostedOs)
+        extraHelixArguments: /p:BrowserHost=$(_hostedOs)
+        scenarios:
+        - buildwasmapps

eng/pipelines/common/templates/wasm-debugger-tests.yml

@@ -0,0 +1,39 @@
+parameters:
+  alwaysRun: false
+  isExtraPlatformsBuild: false
+  platforms: []
+
+jobs:
+
+# Wasm debugger tests - windows
+- template: /eng/pipelines/common/platform-matrix.yml
+  parameters:
+    jobTemplate: /eng/pipelines/common/global-build-job.yml
+    helixQueuesTemplate: /eng/pipelines/libraries/helix-queues-setup.yml
+    buildConfig: Release
+    runtimeFlavor: mono
+    platforms: ${{ parameters.platforms }}
+    variables:
+      # map dependencies variables to local variables
+      - name: wasmdebuggertestsContainsChange
+        value: $[ dependencies.evaluate_paths.outputs['SetPathVars_wasmdebuggertests.containsChange'] ]
+      - name: alwaysRunVar
+        value: ${{ parameters.alwaysRun }}
+    jobParameters:
+      testGroup: innerloop
+      isExtraPlatforms: ${{ parameters.isExtraPlatformsBuild }}
+      nameSuffix: Mono_DebuggerTests
+      buildArgs: -s mono+libs+libs.tests -c $(_BuildConfig) /p:ArchiveTests=true /p:TestWasmDebuggerTests=true /p:TestAssemblies=false /p:BrowserHost=$(_hostedOs)
+      timeoutInMinutes: 180
+      condition: >-
+        or(
+          eq(variables['alwaysRunVar'], true),
+          eq(dependencies.evaluate_paths.outputs['SetPathVars_wasmdebuggertests.containsChange'], true))
+      # extra steps, run tests
+      extraStepsTemplate: /eng/pipelines/libraries/helix.yml
+      extraStepsParameters:
+        creator: dotnet-bot
+        testRunNamePrefixSuffix: Mono_$(_BuildConfig)
+        extraHelixArguments: /p:BrowserHost=$(_hostedOs)
+        scenarios:
+        - wasmdebuggertests

eng/pipelines/common/templates/wasm-library-aot-tests.yml

@@ -0,0 +1,24 @@
+parameters:
+  alwaysRun: false
+  extraBuildArgs: ''
+  extraHelixArgs: ''
+  isExtraPlatformsBuild: false
+  nameSuffix: ''
+  platforms: []
+  runAOT: false 
+  runSmokeOnlyArg: ''
+
+jobs:
+
+#
+# Build for Browser/wasm, with EnableAggressiveTrimming=true
+#
+- template: /eng/pipelines/common/templates/wasm-library-tests.yml
+  parameters:
+    platforms: ${{ parameters.platforms }}
+    nameSuffix: ${{ parameters.nameSuffix }}
+    isExtraPlatforms: ${{ parameters.isExtraPlatformsBuild }}
+    extraBuildArgs: /p:EnableAggressiveTrimming=true /p:BuildAOTTestsOnHelix=true /p:RunAOTCompilation=${{ parameters.runAOT }} ${{ parameters.extraBuildArgs }}
+    extraHelixArgs: /p:NeedsToBuildWasmAppsOnHelix=true ${{ parameters.extraHelixArgs }}
+    alwaysRun: ${{ parameters.alwaysRun }}
+    runSmokeOnlyArg: $(_runSmokeTestsOnlyArg)

eng/pipelines/common/templates/wasm-library-tests.yml

@@ -0,0 +1,51 @@
+parameters:
+  alwaysRun: false
+  extraBuildArgs: ''
+  extraHelixArgs: ''
+  isExtraPlatformsBuild: false
+  nameSuffix: ''
+  platforms: []
+  runSmokeOnlyArg: ''
+  scenarios: ['normal']
+
+jobs:
+
+#
+# Build for Browser/wasm and test it
+#
+- template: /eng/pipelines/common/platform-matrix.yml
+  parameters:
+    jobTemplate: /eng/pipelines/common/global-build-job.yml
+    helixQueuesTemplate: /eng/pipelines/libraries/helix-queues-setup.yml
+    buildConfig: Release
+    runtimeFlavor: mono
+    platforms: ${{ parameters.platforms }}
+    variables:
+      # map dependencies variables to local variables
+      - name: librariesContainsChange
+        value: $[ dependencies.evaluate_paths.outputs['SetPathVars_libraries.containsChange'] ]
+      - name: monoContainsChange
+        value: $[ dependencies.evaluate_paths.outputs['SetPathVars_mono.containsChange'] ]
+      - name: alwaysRunVar
+        value: ${{ parameters.alwaysRun }}
+    jobParameters:
+      isExtraPlatforms: ${{ parameters.isExtraPlatformsBuild }}
+      testGroup: innerloop
+      nameSuffix: LibraryTests${{ parameters.nameSuffix }}
+      buildArgs: -s mono+libs+host+packs+libs.tests -c $(_BuildConfig) /p:ArchiveTests=true /p:BrowserHost=$(_hostedOs) ${{ parameters.runSmokeOnlyArg }} ${{ parameters.extraBuildArgs }}
+      timeoutInMinutes: 180
+      # always run for runtime-wasm builds (triggered manually)
+      # Always run for rolling builds
+      # Else run on path changes
+      condition: >-
+        or(
+          eq(variables['alwaysRunVar'], true),
+          eq(dependencies.evaluate_paths.outputs['SetPathVars_libraries.containsChange'], true),
+          eq(dependencies.evaluate_paths.outputs['SetPathVars_mono.containsChange'], true))
+      # extra steps, run tests
+      extraStepsTemplate: /eng/pipelines/libraries/helix.yml
+      extraStepsParameters:
+        creator: dotnet-bot
+        testRunNamePrefixSuffix: Mono_$(_BuildConfig)
+        extraHelixArguments: /p:BrowserHost=$(_hostedOs) ${{ parameters.runSmokeOnlyArg }} ${{ parameters.extraHelixArgs }}
+        scenarios: ${{ parameters.scenarios }}

eng/pipelines/common/templates/wasm-runtime-tests.yml

@@ -0,0 +1,39 @@
+parameters:
+  alwaysRun: false
+  isExtraPlatformsBuild: false
+  platforms: []
+
+jobs:
+
+#
+# Build the whole product using Mono and run runtime tests
+#
+- template: /eng/pipelines/common/platform-matrix.yml
+  parameters:
+    jobTemplate: /eng/pipelines/common/global-build-job.yml
+    helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
+    buildConfig: Release
+    runtimeFlavor: mono
+    platforms: ${{ parameters.platforms }}
+    variables:
+      - name: alwaysRunVar
+        value: ${{ parameters.alwaysRun }}
+      - name: timeoutPerTestInMinutes
+        value: 10
+      - name: timeoutPerTestCollectionInMinutes
+        value: 200
+    jobParameters:
+      testGroup: innerloop
+      isExtraPlatforms: ${{ parameters.isExtraPlatformsBuild }}
+      nameSuffix: AllSubsets_Mono_RuntimeTests
+      buildArgs: -s mono+libs -c $(_BuildConfig)
+      timeoutInMinutes: 180
+      condition: >-
+        or(
+          eq(variables['alwaysRunVar'], true),
+          eq(dependencies.evaluate_paths.outputs['SetPathVars_mono.containsChange'], true),
+          eq(dependencies.evaluate_paths.outputs['SetPathVars_runtimetests.containsChange'], true))
+      extraStepsTemplate: /eng/pipelines/common/templates/runtimes/wasm-runtime-and-send-to-helix.yml
+      extraStepsParameters:
+        creator: dotnet-bot
+        testRunNamePrefixSuffix: Mono_$(_BuildConfig)