dotnet restore hangs indefinitely

[davkean]

From @richlander on July 3, 2017 20:57

See:

• .NET Core 2.0 Preview 2 dotnet/core#711 (comment)
• .NET Core 2.0 Preview 2 dotnet/core#711 (comment)

/cc @rrelyea

Copied from original issue: dotnet/core#724

[davkean]

From @MathiasKowoll on July 3, 2017 22:45

I looked that while having many libs in the folder wwwroot\lib dotnet restore takes a while to restore the files, If I remove that folder it restores immediately.

This happens with al dotnet commands. CLI and Visual Studio 2017 Preview 3

[davkean]

From @dsyme on July 4, 2017 22:22

I put a full process dump of a hung dotnet.exe here: https://1drv.ms/u/s!Aq5gXOPBI5HdiroRLPy1hDY3yK80kQ

C:\GitHub\misc>dotnet --version
2.0.0-preview2-006497

[davkean]

From @dsyme on July 4, 2017 23:5

I eventually realised that I had a very large number of files in the subdirectories under the c:\misc in which I was running "dotnet". The c:\misc directory itself contained just the results of a dotnet new. The subdirectories included Git repos and much more - 20,000 files and 1.6GB

Now, it's my belief that the presence of these files shouldn't have caused problems for a simple dotnet restore but it seems they did.

I tried to procmon.exe the activity of dotnet.exe but the trace was large. The screenshot below showed that it is indeed accessing all the subdirectories, In this case, it seemed like the thing just never terminated, I don't know what it was doing though o all those blocked threads

Anyway, I think the bug here is that running dotnet.exe in a folder containing 1GB of misc files in subdirectories either takes a very long time or simply doesn't terminate

Now I know that I can get on with the rest of my work :)

[davkean]

From @Petermarcu on July 7, 2017 7:43

@rrelyea should this get moved to the nuget repo to get tracked?

[davkean]

From @Lanayx on July 7, 2017 9:1

I got the same issue. For me the problem was that new dotnet restore (from dotnet-sdk-2.0.0-preview2-006497-win-x64) ignores disabledPackageSources configuration in NuGet.Config, and I had some of them unreachable. Removing these sources helped.

[davkean]

From @richlander on July 7, 2017 13:53

@Petermarcu I was going to make @ https://github.com/NuGet/NuGet.Client but issues are not enabled.

[davkean]

From @Petermarcu on July 7, 2017 15:24

@richlander you make it on the nuget/home repo

[davkean]

From @forki on July 7, 2017 18:0

Same issue here: fable-compiler/Fable#1042

[davkean]

From @rrelyea on July 7, 2017 18:22

Working on creating NuGet/home issues ... will link when created.
Update: Actually, I lied...let's drill into these 2 issues a bit first...

[davkean]

From @rrelyea on July 7, 2017 18:24

@dsyme - in dotnet cli v1 previews, we used to walk all the sub directories. I had thought we changed that when we switched to make restore recursive (across project references) by defeault. do you have the same problem if you do a "dotnet restore --no-dependencies" in that same directory?

[davkean]

From @rrelyea on July 7, 2017 18:25

@MathiasKowoll - can you please give me repro steps for your problem (full details), and how you avoided it.

[davkean]

From @rrelyea on July 7, 2017 18:34

@forki - I am unable to repro.
after you have done dotnet new, a restore should have already been completed.
when you do a build, it will also do a restore, if it deems that one is necessary.
Can you please file an issue in NuGet/Home about the issue you are seeing, with repro steps (including setup steps if necessary?

[davkean]

From @MathiasKowoll on July 7, 2017 18:49

@rrelyea create mvc app with individual authentication.

replace bower.sjon with the following.

{
"name": "ItsomaxCms",
"private": true,
"dependencies": {
"jquery": "^3.2.1",
"jquery-ui": "1.12.1",
"bootstrap": "^3.3.7",
"font-awesome": "4.3.0",
"jquery-validation": "1.16.0",
"jquery-validation-unobtrusive": "3.2.6",
"datatables.net": "^1.10.15",
"datatables.net-bs": "^2.1.1",
"datatables.net-buttons": "^1.3.1",
"datatables.net-buttons-bs": "^1.3.1",
"datatables.net-fixedheader": "^3.1.2",
"datatables.net-fixedheader-bs": "^3.1.2",
"datatables.net-keytable": "^2.2.1",
"datatables.net-responsive": "^2.1.1",
"datatables.net-responsive-bs": "^2.1.1",
"datatables.net-scroller": "^1.4.2",
"datatables.net-scroller-bs": "^1.4.2",
"pdfmake": "^0.1.29",
"echarts": "^3.6.1",
"smartwizard": "latest",
"fastclick": "latest",
"nprogress": "latest",
"ionicons": "latest",
"slim-scroll": "latest",
"icheck": "latest",
"switchery": "latest",
"bootstrap-table": "latest",
"tableExport.jquery.plugin": "latest",
"x-editable": "latest"
},
"resolutions": {
"jspdf": "1.1.239 || 1.3.2"
}
}

make bower install

run dotnet restore.

[davkean]

From @forki on July 7, 2017 18:51

@rrelyea thanks for looking into this. I notified the original author of the issue. We have multiple people reporting this on the fable gitter chat. I assume it's related to node_modules and dotnet restore iterating long time over it

[davkean]

From @forki on July 7, 2017 19:24

Seems that things worked with preview1 see fable-compiler/Fable#1042

[forki]

I don't think it's nuget issue since we also see slow behavior in dotnet build.

[mikkeljohnsen]

If I install "kendo-ui-core" through bower "dotnet build" will hang indefinitely.

If I delete "wwwroot/lib/kendo-ui-core/" it will build again.

Please fix this ASAP, so we can get on with development.

[mishra14]

Tagging @emgarten for restore visibility.

@forki @mikkeljohnsen : Are you seeing an indefinite hang in dotnet restore as well?

[mikkeljohnsen]

@mishra14 dotnet restore is not indefinitely but takes about 5 minutes.
and dotnet build is also not indefinitely from the command line it takes about 5 minutes as well.
(did not wait that long before)

[forki]

The same is happening on azure now. This issue spreads.dotnet/core#724 (comment)

[emgarten]

This appears to be an issue with msbuild file globbing in the project on dotnet CLI. MSBuild in desktop does not have this issue. I also don't see any changes to the SDK around the props including these.

Using the bower.json file above here are the numbers I am getting:

Scenario	Time
dotnet msbuild /t:clean	56s
dotnet build --no-restore	60s
dotnet restore	167s
dotnet restore /p:EnableDefaultItems=false	2s
(desktop) msbuild /t:restore	3s

Restore appears to make the existing problem worse, there may be a target that is changing properties and causing things to be evaluated additional times. I'll take a look at what can be improved there. The main issue appears to be the time globbing takes, the project should not take this long to evaluate just to return the project and package references that restore needs.

Workaround

dotnet restore /p:EnableDefaultItems=false

EnableDefaultItems=false will disable globbing for project directory files during restore.

[mikkeljohnsen]

EnableDefaultItems=false. Will also disable all items in VS (Mac). So not really useful.

And VS (Mac 7.1 build 1281) Still hangs on "Run" (Build)

[emgarten]

@mikkeljohnsen EnableDefaultItems=false should only be used with restore where the items are not needed. For build you will of course still need them, there you can run with --no-restore to speed things up.

[forki]

@emgarten is there a fix to see on the horizon?

[emgarten]

@forki not yet, it is still being investigated. I'll keep you posted.

[mikkeljohnsen]

@emgarten "dotnet build --no-restore" still takes more than 1 minute for a simple project with kendo-ui-core installed. And from VS (Mac) it is not able to compile at all.

Hope there will be a fix very soon. Because only solution is to downgrade to preview1

[dsplaisted]

I think I've figured out what is causing this, and you should be able to workaround it for now by setting the SetLinkMetadataAutomatically property to false in your projects.

[emgarten]

@dsplaisted is there an issue tracking the work needed for this? Is it going to be on the SDK side?

[mikkeljohnsen]

@dsplaisted Thanks that works.

[emgarten]

This issue was moved to dotnet/sdk#1434