Antlr4Compile fails on TeamCity 10.0.5 w/ Microsoft Build Tools 2017: Could not load file or assembly Antlr4BuildTasks

[wjrogers]

I have converted one of my projects to the new project.json nuget reference format and upgraded the Antlr4 package to 4.6.1-beta002. The latter fixed the initial build issues I encountered in Visual Studio, but I'm still having trouble building this project in TeamCity:

[Antlr4Compile] C:\Users\Services.nuget\packages\antlr4.codegenerator\4.6.1-beta002\build\Antlr4.CodeGenerator.targets(110, 5): error MSB4062: The "Antlr4.Build.Tasks.Antlr4ClassGenerationTask" task could not be loaded from the assembly Antlr4BuildTasks, Version=4.0.0.0, Culture=neutral, PublicKeyToken=eb42632606e9261f. Could not load file or assembly 'Antlr4BuildTasks, Version=4.0.0.0, Culture=neutral, PublicKeyToken=eb42632606e9261f' or one of its dependencies. The system cannot find the file specified. Confirm that the declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask.

The assembly is present in the same folder: C:\Users\Services\.nuget\packages\antlr4.codegenerator\4.6.1-beta002\build\Antlr4BuildTasks.dll

The build server has Microsoft Build Tools 2017 installed as well as the SDKs and targeting packs for .NET Framework 4.5.1, 4.6.1, and 4.6.2. Any ideas about what I might be missing?

[wjrogers]

Here's an assembly binder log. It looks like it's trying to load the task from the wrong path?

*** Assembly Binder Log Entry  (3/21/2017 @ 11:41:31 AM) ***

The operation failed.
Bind result: hr = 0x80070002. The system cannot find the file specified.

Assembly manager loaded from:  C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
Running under executable  C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\bin\MSBuild.exe
--- A detailed error log follows. 

=== Pre-bind state information ===
LOG: DisplayName = Antlr4BuildTasks, Version=4.0.0.0, Culture=neutral, PublicKeyToken=eb42632606e9261f
 (Fully-specified)
LOG: Appbase = file:///C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/MSBuild/15.0/bin/
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = NULL
LOG: AppName = MSBuild.exe
Calling assembly : Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a.
===
LOG: This bind starts in default load context.
LOG: Using application configuration file: C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\bin\MSBuild.exe.Config
LOG: Using host configuration file: 
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.
LOG: Post-policy reference: Antlr4BuildTasks, Version=4.0.0.0, Culture=neutral, PublicKeyToken=eb42632606e9261f
LOG: GAC Lookup was unsuccessful.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/MSBuild/15.0/bin/Antlr4BuildTasks.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/MSBuild/15.0/bin/Antlr4BuildTasks/Antlr4BuildTasks.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/MSBuild/15.0/bin/Antlr4BuildTasks.EXE.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/MSBuild/15.0/bin/Antlr4BuildTasks/Antlr4BuildTasks.EXE.
LOG: All probing URLs attempted and failed.

[wjrogers]

I have also tried changing nuget restore to use NuGet 4.0.0. This produces somewhat different output in project.lock.json:

--- 3.5.0/project.lock.json     2017-03-16 16:21:37.715825400 -0400
+++ 4.0.0/project.lock.json     2017-03-16 13:20:36.827199400 -0400
@@ -1,15 +1,23 @@
 {
-  "version": 1,
+  "version": 2,
   "targets": {
     ".NETFramework,Version=v4.5.1": {
       "Antlr4/4.6.1-beta002": {
+        "type": "package",
         "dependencies": {
           "Antlr4.CodeGenerator": "4.6.1-beta002",
           "Antlr4.Runtime": "4.6.1-beta002"
         }
       },
-      "Antlr4.CodeGenerator/4.6.1-beta002": {},
+      "Antlr4.CodeGenerator/4.6.1-beta002": {
+        "type": "package",
+        "build": {
+          "build/Antlr4.CodeGenerator.props": {},
+          "build/Antlr4.CodeGenerator.targets": {}
+        }
+      },

However, the outcome is worse: msbuild doesn't run the Antlr4Compile task at all. I'm not sure how to troubleshoot further from here.

[wjrogers]

This is starting to look more like a TeamCity bug to me, so I have reported it to JetBrains here: https://youtrack.jetbrains.com/issue/TW-49205

Going to leave this open for now because I'm still not sure of the root cause.

[sebandraos]

It was my understanding that as of the release of VS2017 and the associated tooling that the json project files had now been deprecated. I have encountered similar problems when building with TeamCity with the json setup in .NetStandard projects but had put it down to the aforementioned issue.

[wjrogers]

I'm talking about project.json the NuGet replacement for packages.config, not project.json the .NET Core project file format. The latter was deprecated and subsequently removed from the tooling. The NuGet project.json, however, is still documented as the official new format for specifying NuGet package dependencies in .NET projects:

The project.json file replaces packages.config in UWP, ASP.NET 5, and .NET desktop projects (WPF and WinForms) as a means to maintain the list of packages used in a project. (For .NET Core, refer to NET Core Tools - project.json).

https://docs.microsoft.com/en-us/nuget/schema/project-json

I believe NuGet project.json will be deprecated and replaced by the PackageReference MSBuild element, but the tooling isn't ready, so for now, this is the official thing.

All that having been said, the inner workings of project.json and PackageReference are very similar: they output per-project intermediate files that get picked up by the build process to discover information about NuGet references. These files' paths and content differ slightly, but the overall idea is the same: stop putting redundant information in the main project file when the package manager can supply it at build time.

• "lockfile" listing the full, resolved dependency graph
• .props defining NuGet-supplied build properties and importing any NuGet package-supplied properties (e.g. Antlr4.CodeGenerator.props)
• .targets defining NuGet-supplied build targets and importing any NuGet package-supplied targets (e.g. Antlr4.CodeGenerator.targets)

For some reason, on TeamCity, using NuGet 3.5.0's project.json reference format, MSBuild doesn't know about Antlr4.CodeGenerator.props. That's the problem I'm trying to figure out here.

[lextm]

@wjrogers you probably will soon say goodbye to project.json for NuGet, as PackageReference with MSBuild 15 is clearly the future,

http://blog.nuget.org/20170316/NuGet-now-fully-integrated-into-MSBuild.html

So IMHO, for all VS releases before VS2017, developers should just use packages.config. For VS2017 and above, developers should use PackageReference. We can safely live without project.json for good.

[wjrogers]

@lextm like I said in my previous post, PackageReference isn't stable enough yet (see for example NuGet/Home#4532). I would like to enjoy the benefit of the new reference workflow now. The NuGet documentation says project.json is the reference file format for NuGet 3.x. It is working correctly for me in every scenario except for the one in this issue. I am pretty sure we can figure out why!

[tsutomi]

I have a similar situation that I suspect has the same origin of this case, although it's happening on AppVeyor and not in TeamCity (not sure what infrastructure AppVeyor uses, maybe TeamCity).

In my machines I'm mounting VS2017 (that means I'm using the new MSBuild project) and I am developing the .NET Core version of a SQL database system (DeveelDB Core, DeveelDB).
The ANTLRv4 version I'm using for the .NET regular version is building fine both in my machines and at AppVeyor (but it's developed using a VS2015 version).
The ANTLRv4 version for the .NET Core project is failing on AppVeyor (Log of the Failing Build), but works on my machines.

I tried referencing the NuGet packages Microsoft.Build.Framework, Microsoft.Build.Tasks and Microsoft.Build.Utilities, but it doesn't help.

Any developments on the understanding of the cause of this issue?

[sharwell]

@tsutomi That would be a separate bug related to the use of dotnet tooling.

[wjrogers]

Forgot to update this issue, but I was able to get my build working using PackageReference in old/classic csproj projects with the release of Visual Studio Build Tools 15.1 and NuGet 4.1.0. The tool bug I mentioned earlier has been fixed (NuGet/Home#4532). Doesn't fix the issue reported here, exactly, but at least there's a path forward now.

[tsutomi]

@sharwell can you reference the issue and eventually the solution for the bug? Is it there around any sample VS2017 MSBuild .csproj file that I can use as reference?