[DartLab B2B feature] dropname for base VS should be retrieved using the baseline. #5283

vishavpandhi · 2023-06-27T18:18:07Z

Bug

Fixes: https://github.com/NuGet/Client.Engineering/issues/2380

Regression? Last working version: N/A

Description

@kartheekp-ms noticed that B2B upgrade is disabled in almost 4 PR build logs. When B2B is disabled, DartLab installs VS from scratch instead of upgrading. To enable B2B upgrade, the base VS drop name needs to be retrieved from the baseline. Otherwise, if the base and target VS use the same drop, their versions will be identical, resulting in the disabling B2B upgrade.

PR Checklist

eng/pipelines/templates/Apex_Tests_On_Windows.yml

kartheekp-ms · 2023-07-06T01:23:50Z

eng/pipelines/templates/End_To_End_Tests_On_Windows.yml

    candidateBaselineBuilds: $(BaselineBuildCommitIds)
    visualStudioBootstrapperURI: $(bootstrapperUrl)
    visualStudioInstallationParameters: $(VisualStudio.InstallationUnderTest.SetupParameters)
    testLabPoolName: VS-Platform
    dartLabEnvironment: ${{parameters.DartLabEnvironment}}
    visualStudioSigning: Test
+    testMachineConfigurationJobTimeoutInMinutes: 30


Is this timeout setting required?

yes, because of the additional tasks, it will take ~1-2 minutes extra. But occasionally Azure DevOps might be slow, that's why I have set timeout to 30

vishavpandhi · 2023-07-06T19:57:27Z

Apex tests on Windows failed because of Azure DevOps issue which is not related to the changes in this PR.
To overcome this I have added retries.

kartheekp-ms · 2023-07-06T20:24:17Z

eng/pipelines/templates/Apex_Tests_On_Windows.yml

+        - task: PowerShell@2
+          name: SetVisualStudioBaseBuildID
+          displayName: Set 'VisualStudio.BaseBuild.ID'
+          retryCountOnTaskFailure: 3


checked with @vishavpandhi offline. This parameter only enables retries for the following steps. Apex/End to end tests will not be rerun upon failure.

Get Baseline build commit ids

Set 'VisualStudio.BaseBuild.ID'

Set 'VisualStudio.BaseBuild.ProductsDropName

Set 'VisualStudio.BaseBuild.ProductsDropName' to drop from target build

kartheekp-ms · 2023-07-11T20:56:09Z

eng/pipelines/templates/Apex_Tests_On_Windows.yml

      candidateBaselineBuilds: $(BaselineBuildCommitIds)
      testLabPoolName: VS-Platform
      dartLabEnvironment: ${{parameters.DartLabEnvironment}}
      visualStudioSigning: Test
+      testMachineConfigurationJobTimeoutInMinutes: 30


Related to timeout https://github.com/NuGet/NuGet.Client/pull/5283/files#r1253811132

kartheekp-ms · 2023-07-11T21:04:48Z

eng/pipelines/templates/Apex_Tests_On_Windows.yml

+            targetType: 'inline'
+            script: |
+              $buildDrop = "${{parameters.baseBuildDrop}}" -split "/"
+              $dropName = "Products/DevDiv/VS/$($buildDrop[-2])/$($buildDrop[-1])"


If my understanding is correct, if there is format in the build drop URL then we should tweak these indexes, correct? Do you anticipate any changes to the build drop in the near future?

these changes can be breaking changes, so they aren't generally made.

kartheekp-ms · 2023-07-11T21:09:49Z

The deploy machine step for End to End tests part 1 job for this PR build took only 17 minutes which is a significant improvement. As you can see from the log, Visual Studio was upgraded instead of installing from scratch because of B2B feature.

zivkan · 2023-07-12T08:55:58Z

Why did only 1 of the 3 dartlab jobs get the benefit? They're all requesting the same build of VS.

kartheekp-ms · 2023-07-12T16:44:40Z

Why did only 1 of the 3 dartlab jobs get the benefit? They're all requesting the same build of VS.

For the remaining 2 jobs, DartLab was unable to find a machine in the pool hence it ended up creating a machine for these runs. The following logs also suggest that Visual Studio was upgraded instead of installing from scratch (Deploying artifact 1/3: Upgrade Visual Studio Under Test). The time it took creates the machine caused delay in completing the deploy machine job. Whereas in the best-case scenario, as shown in the screenshot, DartLab found a machine in the pool and hence the job took less time.

No machines were found in the pool that match the checkpoint's filter. Attempting to create another machine for this run...
Successfully created machine 'dtl-kkdxt302' for this run.
[dtl-kkdxt302] Click here to view the machine's activity logs: 
##[debug][dtl-kkdxt302] Updating 8 tag(s)...
##[debug][dtl-kkdxt302] Waiting until machine is ready to deploy artifacts...
[dtl-kkdxt302] Machine is ready to deploy artifacts.
##[debug][dtl-kkdxt302] Deploying 3 artifact(s)...
##[debug][dtl-kkdxt302] Waiting for artifact deployment to begin...
[dtl-kkdxt302] Artifact deployment has successfully began. Click the following link for more details: 
##[debug][dtl-kkdxt302] Waiting for 3 artifact(s) to deploy...
##[debug][dtl-kkdxt302] **Deploying artifact 1/3: Upgrade Visual Studio Under Test**
##[debug][dtl-kkdxt302] Deploying artifact 2/3: Restart Machine
[dtl-kkdxt302] Successfully deployed 3 artifact(s).
##[debug][dtl-kkdxt302] Machine is ready, notifying caller.
##[debug]A deploy machine task has completed, 0 task(s) remaining...
Successfully deployed machine 'dtl-kkdxt302'.
1/1 machine(s) deployed successfully, 0/1 machine(s) still in progress...
Successfully deployed 1 machine(s): dtl-kkdxt302

…d using the baseline. (#5283)" This reverts commit 0cf8fb8.

Update Apex_Tests_On_Windows.yml

3ab6793

vishavpandhi requested a review from a team as a code owner June 27, 2023 18:18

ghost added the Community PRs created by someone not in the NuGet team label Jun 27, 2023

Update Apex_Tests_On_Windows.yml

4d3ce56

kartheekp-ms added the Approved for CI label Jun 27, 2023

kartheekp-ms marked this pull request as draft June 27, 2023 18:32

kartheekp-ms self-assigned this Jun 27, 2023

vishavpandhi added 8 commits June 27, 2023 12:35

increase testMachineConfigurationJobTimeoutInMinutes to 30 minutes

8eccfc8

Update Apex_Tests_On_Windows.yml

24a49c4

Update Apex_Tests_On_Windows.yml

ac343e6

Update Apex_Tests_On_Windows.yml

c520e36

correct syntax

ba803e8

fix drop name

b6f8390

dont run task if variable BaselineBuildCommitIds is not set

499932f

dont throw error

08afa72

vishavpandhi marked this pull request as ready for review July 5, 2023 18:02

kartheekp-ms marked this pull request as draft July 5, 2023 18:13

kartheekp-ms reviewed Jul 5, 2023

View reviewed changes

eng/pipelines/templates/Apex_Tests_On_Windows.yml Show resolved Hide resolved

Update End_To_End_Tests_On_Windows.yml

4c09e26

vishavpandhi changed the title ~~Update Apex_Tests_On_Windows.yml~~ dropname for base VS should be retrieved using the baseline Jul 5, 2023

kartheekp-ms reviewed Jul 6, 2023

View reviewed changes

eng/pipelines/templates/Apex_Tests_On_Windows.yml Show resolved Hide resolved

kartheekp-ms reviewed Jul 6, 2023

View reviewed changes

kartheekp-ms changed the title ~~dropname for base VS should be retrieved using the baseline~~ [DartLab B2B feature] dropname for base VS should be retrieved using the baseline. Jul 6, 2023

kartheekp-ms marked this pull request as ready for review July 6, 2023 01:40

jeffkl previously approved these changes Jul 6, 2023

View reviewed changes

add retries

1234cc7

vishavpandhi dismissed jeffkl’s stale review via 1234cc7 July 6, 2023 19:54

add retries

f848151

jeffkl removed the Approved for CI label Jul 6, 2023

jeffkl added the Approved for CI label Jul 6, 2023

kartheekp-ms reviewed Jul 6, 2023

View reviewed changes

kartheekp-ms reviewed Jul 11, 2023

View reviewed changes

jeffkl approved these changes Jul 11, 2023

View reviewed changes

kartheekp-ms approved these changes Jul 11, 2023

View reviewed changes

kartheekp-ms merged commit 0cf8fb8 into NuGet:dev Jul 13, 2023

zivkan added a commit that referenced this pull request Dec 26, 2023

Revert "[DartLab B2B feature] dropname for base VS should be retrieve…

a9273bc

…d using the baseline. (#5283)" This reverts commit 0cf8fb8.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DartLab B2B feature] dropname for base VS should be retrieved using the baseline. #5283

[DartLab B2B feature] dropname for base VS should be retrieved using the baseline. #5283

vishavpandhi commented Jun 27, 2023 •

edited by kartheekp-ms

Loading

kartheekp-ms Jul 6, 2023

vishavpandhi Jul 6, 2023 •

edited

Loading

vishavpandhi commented Jul 6, 2023

kartheekp-ms Jul 6, 2023

kartheekp-ms Jul 11, 2023

kartheekp-ms Jul 11, 2023

vishavpandhi Jul 11, 2023

kartheekp-ms commented Jul 11, 2023 •

edited

Loading

zivkan commented Jul 12, 2023

kartheekp-ms commented Jul 12, 2023 •

edited

Loading

[DartLab B2B feature] dropname for base VS should be retrieved using the baseline. #5283

[DartLab B2B feature] dropname for base VS should be retrieved using the baseline. #5283

Conversation

vishavpandhi commented Jun 27, 2023 • edited by kartheekp-ms Loading

Bug

Description

PR Checklist

kartheekp-ms Jul 6, 2023

Choose a reason for hiding this comment

vishavpandhi Jul 6, 2023 • edited Loading

Choose a reason for hiding this comment

vishavpandhi commented Jul 6, 2023

kartheekp-ms Jul 6, 2023

Choose a reason for hiding this comment

kartheekp-ms Jul 11, 2023

Choose a reason for hiding this comment

kartheekp-ms Jul 11, 2023

Choose a reason for hiding this comment

vishavpandhi Jul 11, 2023

Choose a reason for hiding this comment

kartheekp-ms commented Jul 11, 2023 • edited Loading

zivkan commented Jul 12, 2023

kartheekp-ms commented Jul 12, 2023 • edited Loading

vishavpandhi commented Jun 27, 2023 •

edited by kartheekp-ms

Loading

vishavpandhi Jul 6, 2023 •

edited

Loading

kartheekp-ms commented Jul 11, 2023 •

edited

Loading

kartheekp-ms commented Jul 12, 2023 •

edited

Loading