[Bug]: Very slow restore or OOM when using NoWarn

[mjolka]

NuGet Product Used

dotnet.exe, NuGet.exe

Product Version

6.1.0.106

Worked before?

No response

Impact

I'm unable to use this version

Repro Steps & Context

Description

A NoWarn in a package reference can add minutes to a restore, or make it fail with an OutOfMemoryException.

This is related to #11222 and NuGet/NuGet.Client#4475.

Background

TransitiveNoWarnUtils performs a breadth-first traversal of the dependency graph.

The first time we find a path from the root node to a project node v, we store the union of the NoWarns seen along the path in a dictionary called seen.

Each subsequent time we find a path from the root node to v, we check if the NoWarns seen along the new path to v are a superset of seen[v].

If that's the case, we don't need to process the node again.

If that's not the case, we update seen[v] to be the intersection of seen[v] and the NoWarns seen along the new path to v.

There is a bug in the method for finding the intersection that causes nodes to be processed more often than they should be. As a consequence, a single NoWarn can add minutes to a project restore, or cause the restore to fail with an OutOfMemoryException.

In the method for checking whether the NoWarns along one path are a superset of the NoWarns along another path (NodeWarningProperties.IsSubSetOf), null is considered to be the empty set.

The bug is this: in the method to get the intersection of the NoWarns (NodeWarningProperties.GetIntersect), null doesn't act like the empty set.

Steps to reproduce

To reproduce this issue, create n >= 30 projects - where ClassLibrary{i} has project references to ClassLibary1..ClassLibrary{i-1}.

In addition, create three more projects: Root, RootLeft, and RootRight with the following project references:

Root -> RootLeft -> ClassLibrary{n}
Root -> RootRight -> ClassLibrary{n}

Add the following two package references to RootLeft:

<PackageReference Include="Microsoft.EntityFrameworkCore" Version="6.0.0">
  <NoWarn>NU1000</NoWarn>
</PackageReference>
<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="5.0.2" />

(Code to create all these projects can be found here: https://github.com/mjolka/LotsOfDependenciesIntersect)

The single nowarn in RootLeft causes the package restore to go from ~3 seconds to ~7 minutes when n = 30.

When n = 35, the restore fails with an OutOfMemoryException.

The reason being, all ClassLibrary{i} projects are found first via a path that goes through RootLeft. This path contains a single NoWarn.

When ClassLibrary{i} is later found via a path that goes through RootRight, these are no NoWarns along the path. The code to calculate the intersection of the NoWarns should result in the empty set. However, because of the bug, after taking the intersection, seen[v] still contains a single NoWarn, and the node will be processed again each time we find a new path to it.

The fix is to align the behaviour of GetIntersect to that of IsSubSetOf by treating null as the empty set. I'm able to contribute a pull request with this change.

Verbose Logs

No response

[nkolev92]

We'd be happy to take a fix for this @mjolka

[wrall]

I see that there hasn't been a PR for this, so I'm submitting one. This probably doesn't have a substantial performance impact - I was not able to reproduce the significant slowness reported in @mjolka's LotsOfDependenciesIntersect solution. Perhaps that was remedied by the fix for #11222.

Edit: it has a performance impact in the form of memory utilization, it just isn't as striking as in #11222.

[bouchraRekhadda]

We seem to have the same issue when using MSBuild 17.2.1 (17.2.1+52cd2da31 for .NET Framework, 32bit) with .NET Core 3.1 projects restore

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Collections.Generic.Queue1.SetCapacity(Int32 capacity) at System.Collections.Generic.Queue1.Enqueue(T item)
at NuGet.Commands.TransitiveNoWarnUtils.AddDependenciesToQueue(IEnumerable1 dependencies, Queue1 queue, HashSet1 projectWideNoWarn, Dictionary2 packageSpecificNoWarn)
at NuGet.Commands.TransitiveNoWarnUtils.ExtractTransitiveNoWarnProperties(RestoreTargetGraph targetGraph, String parentProjectName, HashSet1 parentProjectWideNoWarn, Dictionary2 parentPackageSpecificNoWarn, Dictionary2 warningPropertiesCache) at NuGet.Commands.TransitiveNoWarnUtils.CreateTransitiveWarningPropertiesCollection(IEnumerable1 targetGraphs, PackageSpec parentProjectSpec)
at NuGet.Commands.RestoreCollectorLogger.get_TransitiveWarningPropertiesCollection()
at NuGet.Commands.RestoreCollectorLogger.IsWarningSuppressed(IRestoreLogMessage message)
at NuGet.Commands.RestoreCollectorLogger.Log(IRestoreLogMessage message)
at NuGet.Commands.RestoreCollectorLogger.Log(ILogMessage message)
at NuGet.Commands.LockFileBuilder.CreateLockFile(LockFile previousLockFile, PackageSpec project, IEnumerable1 targetGraphs, IReadOnlyList1 localRepositories, RemoteWalkContext context, LockFileBuilderCache lockFileBuilderCache)
at NuGet.Commands.RestoreCommand.BuildAssetsFile(LockFile existingLockFile, PackageSpec project, IEnumerable1 graphs, IReadOnlyList1 localRepositories, RemoteWalkContext contextForProject)
at NuGet.Commands.RestoreCommand.d__45.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Commands.RestoreRunner.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Commands.RestoreRunner.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Commands.RestoreRunner.d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Commands.RestoreRunner.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Commands.RestoreRunner.d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at NuGet.Build.Tasks.BuildTasksUtility.d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Build.Tasks.RestoreTask.d__43.MoveNext()

In our Directory.Build.props we use NoWran=NU1608 and NoWarn="NU1701" on certain packages and we set

    <CentralPackageTransitivePinningEnabled>false</CentralPackageTransitivePinningEnabled>
    <ManagePackageVersionsCentrally>false</ManagePackageVersionsCentrally>

I tried to snapshot the restore ("C:\Program Files\Microsoft Visual Studio\2022\Professional\Msbuild\Current\Bin\MSBuild.exe" /t:restore /v:m /m) process memory consumption:

Please let me know if you will need further details about our use case.

[bouchraRekhadda]

Hello @erdembayar,
When will the fix be released please ?

[erdembayar]

Hello @erdembayar, When will the fix be released please ?

It's bit too early to say when it would be released. Most likely it would be released with nuget 6.3.

[nkolev92]

@bouchraRekhadda We'll set the Milestone in the issue when we know for sure.

[bouchraRekhadda]

Alright, thank you @nkolev92 and @erdembayar