-
Notifications
You must be signed in to change notification settings - Fork 897
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix native library loading on Linux #1732
Conversation
|
OK - your point is that I think that code has always been broken. However, NerdBank.GitVersioning used a custom AssemblyLoadContext which loaded the native library (and added the Because LibGit2Sharp now uses the NativeLibrary class, that no longer works and the underlying bug is exposed. Does that make some sense? |
Why is this a problem? There used to be code in mono, then .NET Core, to add the |
@ethomson Yes, as far as I know, .NET Core will add the E.g. Furthermore, #1714 changed the native library loading mechanism to use You can see this in https://andrewarnott.visualstudio.com/OSS/_build/results?buildId=1903&view=logs&jobId=084ed6c3-25bc-526e-5414-75f5931b38aa, the code failed to load the native library (and went to the fallback path, which also failed) |
Yes, The |
LibGit2Sharp/Core/NativeMethods.cs
Outdated
foreach (var runtimeFolder in Directory.GetDirectories(Path.Combine(assemblyDirectory, "runtimes"), $"*-{processorArchitecture}")) | ||
var runtimesDirectory = Path.Combine(assemblyDirectory, "runtimes"); | ||
|
||
if(Directory.Exists(runtimesDirectory)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this change unrelated? Does something bad happen with Directory.GetDirectories
when the directory doesn't exist?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, calling Directory.GetDirectories for a directory which doesn't exist throws an exception:
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: The "Nerdbank.GitVersioning.Tasks.GetBuildVersion" task failed unexpectedly. [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: System.TypeInitializationException: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception. [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: ---> System.IO.DirectoryNotFoundException: Could not find a part of the path '/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/MSBuildCore/runtimes'. [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound) [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at System.IO.Enumeration.FileSystemEnumerator`1.Init() [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at System.IO.Enumeration.FileSystemEnumerator`1..ctor(String directory, Boolean isNormalized, EnumerationOptions options) [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized) [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at System.IO.Enumeration.FileSystemEnumerableFactory.UserDirectories(String directory, String expression, EnumerationOptions options) [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at System.IO.Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options) [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at LibGit2Sharp.Core.NativeMethods.ResolveDll(String libraryName, Assembly assembly, Nullable`1 searchPath) [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at System.Runtime.InteropServices.NativeLibrary.LoadLibraryCallbackStub(String libraryName, Assembly assembly, Boolean hasDllImportSearchPathFlags, UInt32 dllImportSearchPathFlags) [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at LibGit2Sharp.Core.NativeMethods.git_libgit2_init() [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at LibGit2Sharp.Core.NativeMethods.InitializeNativeLibrary() [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at LibGit2Sharp.Core.NativeMethods..cctor() [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: --- End of inner exception stack trace --- [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at LibGit2Sharp.Core.NativeMethods.git_libgit2_opts(Int32 option, UInt32 level, String path) [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at LibGit2Sharp.Core.Proxy.git_libgit2_opts_set_search_path(ConfigurationLevel level, String path) [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at LibGit2Sharp.GlobalSettings.SetConfigSearchPaths(ConfigurationLevel level, String[] paths) [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at Nerdbank.GitVersioning.GitExtensions.OpenGitRepo(String pathUnderGitRepo, Boolean useDefaultConfigSearchPaths) [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at Nerdbank.GitVersioning.VersionOracle.Create(String projectDirectory, String gitRepoDirectory, ICloudBuild cloudBuild, Nullable`1 overrideBuildNumberOffset, String projectPathRelativeToGitRepoRoot) [/__w/1/s/lib/lib.csproj]
/home/vsts_azpcontainer/.nuget/packages/nerdbank.gitversioning/3.1.29-beta-g67e296ae86/build/Nerdbank.GitVersioning.targets(63,5): error MSB4018: at Nerdbank.GitVersioning.Tasks.GetBuildVersion.ExecuteInner() [/__w/1/s/lib/lib.csproj]
I assumed the intent of this method was to return IntPtr.Zero
instead of throwing.
That makes sense. Thanks for the information. I'm going to tag @bording in as well for 👀 since this is his expertise and not mine. |
@qmfrederik are you setting |
@tmds I don't think NerdBank.GitVersioning used to set With the recent change that approach no longer works, so I crafted a PR which sets But that didn't work because the The corresponding PR in NB.GV is dotnet/Nerdbank.GitVersioning#400 . |
libgit2sharp/LibGit2Sharp/GlobalSettings.cs Lines 32 to 40 in f8e2d42
So I wonder how you are hitting this on non-Windows platforms. Isn't someone setting |
NerdBank.GitVersioning is (amongst others) consumed as an MSBuild task. This means the native libraries are not in their "default" location (in the In the past, this was done with an That exposed the difference in how the NB.GV Perhaps @AArnott can expain it better, but this is how I understand it works 😄 . At the end of the day, the native library name contains the |
Thanks for explaining @qmfrederik , it makes sense to me. |
I should be able to take a closer look at this soon. I know there's some stuff in this area that needs to be cleaned up / removed after a previous PR made some changes that turned out to not work on non-Windows OSes. |
@bording , gentle ping. Let me know if there's anything else I can do to help. |
I've been holding off looking at this because I knew when I did I was going to find a huge mess. I've started looking at it today, and unfortunately I was right. As best as I can tell, your PR that prompted this one (dotnet/Nerdbank.GitVersioning#400) is likely based on a faulty premise: that The way Then on top of that, we now have the logic introduced in #1714 that only comes into play when LibGit2Sharp is being used on .NET Core 3.0. It's also testing to see if This leads to a very confusing mess of there being different ways of trying to load the native library, and the code is currently far more intertwined than it should be:
Not to mention there is code in there to try and support mono as well, though I'm increasingly of the opinion that is no longer worth it. So where does that leave us? To be honest, I'm going to need to spend more time sorting through all of this, but I'm pretty sure that setting The ultimate goal here is that as a library consumer, you don't have to worry about which underlying native library is required for the OS you're on, and with .NET Core 3.0, it looks like there are finally APIs that can make that possible. That means if you care about using LibGit2Sharp from .NET Core 2.0 MSBuild, you're definitely going to have to have all the RID logic yourself in a custom The key thing to figure out is how to ensure that still works while also moving forward to the new stuff .NET Core 3.0. @qmfrederik Which version of .NET Core were you running when you hit the problems with Nerdbank.GitVersioning? What lead you to thinking Until it's more clear to me exactly which of the myriad code paths are impacted, I'd like to hold off on making any more changes here. |
.NET 3.0. This build https://andrewarnott.visualstudio.com/OSS/_build/results?buildId=1950 shows that NB.GV is failing on Ubuntu 19.04:
Couple of things:
Net, I'll create a new PR which focuses on Does that make sense? |
Aha! This is the missing piece in the puzzle. I was wondering why the AssemblyLoadContext was no longer being used.
If not throwing causes the AssemblyLoadContext to be used, that makes sense to me. |
This is tackled in: #1741 |
Thanks @tmds ! |
@qmfrederik Given that #1741 is merged, I assume it's okay to close this one? |
@bording Up to you. My scenario works, so I don't really need this PR anymore. On the other hand, I still think Feel free to close the PR if other concerns (like the ones you mentioned above) need to be addressed first. |
Given that |
GetGlobalSettingsNativeLibraryPath
is currently broken because it never adds thelib
prefix to the native library name (libgit2
does not contain the lib prefix)ResolveDll
, check whether the runtimes directory exists before listing child directories. In some configurations (e.g. NerdBank.GitVersioning) this directory may not exist, and the code would throw an exception.