Travis-CI mono build started failing on 2018-10-05

[jthelin]

On 2018-10-05 the regular sanity-check cron build of master branch using mono platform on Travis-CI started failing, even though previous builds of master branch have worked OK.

The cron build was just rebuilding the same master HEAD commit a0cbb1d as previous build,
so it looks like something may have changed in the build environment?

Any later builds of PR's / branches fail the same way, as do attempts to rebuild the master branch.

Error summary:

The "Microsoft.Build.Tasks.Git.LocateRepository" task failed unexpectedly. 
System.TypeInitializationException: The type initializer for 'Microsoft.Build.Tasks.Git.TaskImplementation' threw an exception.
---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Build.Tasks.Git.Operations' or one of its dependencies 

First failing build:
https://travis-ci.org/jthelin/ServerHost/builds/437876870

Last known good build:
https://travis-ci.org/jthelin/ServerHost/builds/436444986

Failure mode:

/home/travis/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63127-02/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018: The "Microsoft.Build.Tasks.Git.LocateRepository" task failed unexpectedly. [/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj]
/home/travis/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63127-02/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018: System.TypeInitializationException: The type initializer for 'Microsoft.Build.Tasks.Git.TaskImplementation' threw an exception. ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Build.Tasks.Git.Operations' or one of its dependencies [/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj]
/home/travis/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63127-02/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:   at System.AppDomain.Load (System.Reflection.AssemblyName assemblyRef, System.Security.Policy.Evidence assemblySecurity) [0x000e0] in <2943701620b54f86b436d3ffad010412>:0  [/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj]
/home/travis/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63127-02/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:   at (wrapper remoting-invoke-with-check) System.AppDomain.Load(System.Reflection.AssemblyName,System.Security.Policy.Evidence) [/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj]
/home/travis/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63127-02/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:   at System.AppDomain.Load (System.Reflection.AssemblyName assemblyRef) [0x00000] in <2943701620b54f86b436d3ffad010412>:0  [/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj]
/home/travis/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63127-02/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:   at (wrapper remoting-invoke-with-check) System.AppDomain.Load(System.Reflection.AssemblyName) [/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj]
/home/travis/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63127-02/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:   at System.Reflection.Assembly.Load (System.Reflection.AssemblyName assemblyRef) [0x00005] in <2943701620b54f86b436d3ffad010412>:0  [/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj]
/home/travis/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63127-02/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:   at Microsoft.Build.Tasks.Git.TaskImplementation..cctor () [0x0006b] in <77223f90773a4c0e88a42a7f38595356>:0  [/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj]
/home/travis/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63127-02/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:    --- End of inner exception stack trace --- [/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj]
/home/travis/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63127-02/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:   at Microsoft.Build.Tasks.Git.LocateRepository.Execute () [0x0000f] in <77223f90773a4c0e88a42a7f38595356>:0  [/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj]
/home/travis/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63127-02/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute () [0x00023] in <63bca7756d124ea392a81ca1b2d26b9c>:0  [/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj]
/home/travis/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63127-02/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:   at Microsoft.Build.BackEnd.TaskBuilder+<ExecuteInstantiatedTask>d__26.MoveNext () [0x001f6] in <63bca7756d124ea392a81ca1b2d26b9c>:0  [/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj]

Done Building Project "/home/travis/build/jthelin/ServerHost/ServerHost/ServerHost.csproj" (default targets) -- FAILED.

[jthelin]

This may be a related issues:
dotnet/roslyn#29289

[jthelin]

build-good.log

build-bad.log

[jthelin]

Well it appears that the version of mono latest recently changed from 5.14 to 5.16, and that seems to be what is causing this build-break issue.

Locking the build down to use 5.14 explicitly (instead of using latest which now means 5.16) has fixed my broken build of the stable master branch.
https://travis-ci.org/jthelin/ServerHost/builds/439402129

It's not exactly clear which change in mono v5.16 is causing this issue, but here are the release notes:
https://www.mono-project.com/docs/about-mono/releases/5.16.0/

[jthelin]

Fixed by PR #49