Sourcelink doesn't work on Fedora 27

[leecow]

Working with the dotnetsay sample on F27 and hit the following-

$dotnet pack -c release -o nupkg
Microsoft (R) Build Engine version 15.7.177.53362 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.
 
  Restoring packages for /media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj...
  Installing Microsoft.Build.Tasks.Git 1.0.0-beta-62909-01.
  Installing Microsoft.SourceLink.Common 1.0.0-beta-62909-01.
  Installing Microsoft.SourceLink.GitHub 1.0.0-beta-62909-01.
  Generating MSBuild file /media/psf/Home/github/core/samples/dotnetsay/obj/dotnetsay.csproj.nuget.g.props.
  Generating MSBuild file /media/psf/Home/github/core/samples/dotnetsay/obj/dotnetsay.csproj.nuget.g.targets.
  Restore completed in 9.12 sec for /media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj.
  You are working with a preview version of the .NET Core SDK. You can define the SDK version via a global.json file in the current project. More at https://go.microsoft.com/fwlink/?linkid=869452
/home/leecow/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-62909-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018: The "Microsoft.Build.Tasks.Git.LocateRepository" task failed unexpectedly. [/media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj]
/home/leecow/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-62909-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018: System.TypeInitializationException: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception. ---> System.DllNotFoundException: Unable to load shared library 'git2-6311e88' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libgit2-6311e88: cannot open shared object file: No such file or directory [/media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj]
/home/leecow/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-62909-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:    at LibGit2Sharp.Core.NativeMethods.git_libgit2_init() [/media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj]
/home/leecow/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-62909-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:    at LibGit2Sharp.Core.NativeMethods.InitializeNativeLibrary() [/media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj]
/home/leecow/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-62909-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:    at LibGit2Sharp.Core.NativeMethods..cctor() [/media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj]
/home/leecow/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-62909-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:    --- End of inner exception stack trace --- [/media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj]
/home/leecow/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-62909-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:    at LibGit2Sharp.Core.NativeMethods.git_buf_free(GitBuf buf) [/media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj]
/home/leecow/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-62909-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:    at LibGit2Sharp.Core.Proxy.ConvertPath(Func`2 pathRetriever) [/media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj]
/home/leecow/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-62909-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:    at LibGit2Sharp.Repository.Discover(String startingPath) [/media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj]
/home/leecow/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-62909-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:    at Microsoft.Build.Tasks.Git.LocateRepository.Execute() in /_/src/Microsoft.Build.Tasks.Git/LocateRepository.cs:line 19 [/media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj]
/home/leecow/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-62909-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [/media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj]
/home/leecow/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-62909-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [/media/psf/Home/github/core/samples/dotnetsay/dotnetsay.csproj]

[tmat]

This particular issue is fixed in version 1.0.0-beta-62925-02.

[richlander]

Did you get it working on Fedora 27? Last I saw, it still wasn't working with the -02 version either.

[tmat]

Finding libgit2-6311e88.so itself is fixed but apparently now it fails to find libcurl-gnutls.so.4

[tmat]

@bording Any known issues with loading libgit2 on Fedora 27? Seems like a dependency is missing.

[bording]

@tmat The linux library that currently ships in the LibGit2Sharp.NativeBinaries package is built against Ubuntu, so that's the only distro that is likely to work out of the box.

However, I've done work in libgit2/libgit2sharp.nativebinaries#51 to ship binaries for every distro supported by .NET Core. I just need @ethomson to find some time to review it!

[ctaggart]

@bording How are you collecting all the binaries into a single nupkg? It looks that that PR was merged. Any chance ya'll can publish a new version to https://www.nuget.org/packages/LibGit2Sharp.NativeBinaries/ ? It looks we need something newer than 1.0.217.

[bording]

@ctaggart My plan is to finish up things this weekend so we can get a new 0.26.0 preview of LibGit2Sharp next week.

[ctaggart]

Looks like 1.0.226 may native binaries compatible with Fedora:

[bording]

@ctaggart That's correct, but you'll need to wait for a new 0.26.0 prerelease of LibGit2Sharp that uses it.

I'm hoping we'll be able to get that out soon!

[ctaggart]

Oh, I see. This project is locked to LibGit2Sharp 0.26.0-preview-0017 which is locked to LibGit2Sharp.NativeBinaries 1.0.210.

https://github.com/dotnet/sourcelink/blob/master/eng/Versions.props#L17
https://www.nuget.org/packages/LibGit2Sharp/0.26.0-preview-0017

[bording]

Yes, LibGit2Sharp itself is locked to a specific version of the native binaries package, to ensure all the P/Invokes are to the expected version of native library.

[ethomson]

New prerelease is at : https://www.nuget.org/packages/LibGit2Sharp/0.26.0-preview-0027

[tmat]

@ethomson Awesome! I'll update SourceLink and see if we can start running tests on more distros.

[bording]

@tmat Let me know if you need any help figuring out which native library to map to for a given distro. Ideally you'd want something that does the actual RID graph search similar to what I added for libgit2/libgit2sharp#1571, before learning that the DllImports didn't work the way were were hoping they would on non-Windows...

[ctaggart]

@tmat build 1.0.0-beta-63001-01 doesn't work on the microsoft/dotnet:2.1-sdk docker image which is Debian 9.

: error MSB4018: The "Microsoft.Build.Tasks.Git.LocateRepository" task failed unexpectedly. [/build/VMware.vSphere.Management/VMware.vSphere.Management.csproj]
/root/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63001-01/build/Microsoft.Build.Tasks.Git.targets(20,5): error MSB4018: System.TypeInitializationException: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception. ---> System.DllNotFoundException: Unable to load shared library '/root/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63001-01/build/../tools/netcoreapp2.0/runtimes/linux-x64/native/libgit2-6311e88.so' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libssl.so.1.0.0: cannot open shared object file: No such file or directory [/build/VMware.vSphere.Management/VMware.vSphere.Management.csproj]

[bording]

@tmat is going to have to add special logic to load the correct native library before it's going to work, as I mentioned in #66 (comment).

It needs to load the debian.9-x64 library, but it's still loading linux-x64.

[tmat]

@bording I figured we can avoid parsing the JSON file at runtime and instead pre-generate the RID -> directory name map for the platforms that libgit2 supports.

See b6271ac

[tmat]

@bording
Turns out not everything works well on all platforms.

On OSX10.12 calling

Repository.Discover("/tmp/RoslynTests/367fd4f0-8af8-4994-aaa0-c629c015a915")

returns /private/tmp/RoslynTests/367fd4f0-8af8-4994-aaa0-c629c015a915/.git/.

Any idea what's going on?

[bording]

What were you expecting the value to be? The path without /private?

What you're seeing there is that /tmp on macOS is a symlink to /private/tmp.

[tmat]

Yes, I was expecting a path without /private. Why would libgit replace the symlink by its target path?

[ethomson]

First, it’s libgit2. If you’re asking for the canonical path to a git repository, we return what the operating system tells us is canonical.

[tmat]

Canonical path doesn't necessarily mean one with resolved symlinks.

.NET API Path.GetFullPath does not expand symlinks hence the mismatch between the path I get from libgit2 and the one that the tests expect. There is currently no x-plat API on .NET to deal with symlinks (https://github.com/dotnet/corefx/issues/26310), so for now I'm gonna hard-code prefixing /tmp with /private on OSX in the test infrastructure.

[tmat]

We are now testing SourceLink end-to-end scenarios on the following platforms:

• Windows
• Ubuntu 16.04
• CentOS7.1
• Debian8.2
• RHEL7.2
• OSX10.12