System.Buffers, System.Runtime.CompilerServices.Unsafe, etc cannot be loaded

[jstedfast]

Describe the bug

I keep getting reports of my MimeKit and MailKit libraries throwing Exceptions trying to load System.Buffers and System.Runtime.CompilerServices.Unsafe.

Some examples:

• System.Runtime.CompilerServices.Unsafe
  • Getting exception sending emails with MimeKit, after upgrading from 3.4.2 to 3.4.3 jstedfast/MimeKit#872
  • Mailkit 3.4.3 / MsgReader 4.4.x compatibility jstedfast/MailKit#1495
  • Could not load file or assembly 'System.Runtime.CompilerServices.Unsafe, Version=4.0.4.1 jstedfast/MailKit#1462
• System.Buffers:
  • System.BadImageFormatException: Could not load file or assembly 'System.Buffers, Version=4.0.3.0..." jstedfast/MailKit#1524
  • https://stackoverflow.com/questions/75690923/mailkit-mimekit-looking-for-system-buffers-4-0-2-0

To Reproduce

I haven't been able to reproduce it myself, but apparently, it's enough to add the MimeKit nuget package to a project and then create a new MimeMessage:

Install-Package MimeKit

void Main()
{
    var message = new MimeKit.MimeMessage ();
}

Exceptions (if any)

System.IO.FileLoadException
HResult=0x80131040
Message=Could not load file or assembly 'System.Runtime.CompilerServices.Unsafe, Version=4.0.4.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
Source=System.Memory
StackTrace:
at System.Span`1..ctor(T[] array)
at MimeKit.Utils.ValueStringBuilder..ctor(Int32 initialCapacity)
at MimeKit.Utils.Rfc2047.Encode(FormatOptions options, Encoding charset, String text, Boolean phrase)
at MimeKit.Header.EncodeUnstructuredHeader(ParserOptions options, FormatOptions format, Encoding encoding, String field, String value)
at MimeKit.Header.EncodeAddressHeader(ParserOptions options, FormatOptions format, Encoding encoding, String field, String value)
at MimeKit.Header.FormatRawValue(FormatOptions format, Encoding encoding, String value)
at MimeKit.Header.SetValue(FormatOptions format, Encoding encoding, String value)
at MimeKit.Header..ctor(Encoding encoding, HeaderId id, String value)
at MimeKit.HeaderList.set_Item(HeaderId id, String value)
at MimeKit.MimeMessage..ctor()
at SharpLibTestApp.MailKitForm.button1_Click(Object sender, EventArgs e) in C:\Users\Jamal\Documents\Visual Studio 2022\Projects\SharpLib\SharpLibTestApp\MailKitForm.cs:line 75
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr dwComponentID, Int32 reason, Int32 pvLoopData)
at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
at LimoWizCSharpLibTestApp.Program.Main() in C:\Users\Jamal\Documents\Visual Studio 2022\Projects\SharpLib\CSharpLibTestApp\Program.cs:line 25

Further technical details

• Include the output of dotnet --info
• The IDE (VS / VS Code/ VS4Mac) you're running on, and its version

[dsplaisted]

@ericstj @joperezr Do we have any guidance on binding redirects and different versions for packages like System.Runtime.CompilerServices.Unsafe? Looks like users of MimeKit and MailKit are running into this.

[joperezr]

Our guidance for end applications is usually to use PackageReference if they are not in order to have RAR be able to see transitive dependencies, and to make sure they have auto-binding redirects turned on (which are by default). I suppose main issue here is that IN the case of MimeKit and MailKit they are libraries, not end applications, which don't have binding redirects on by default. One advise that could help would be to a) cross-compile for a .NET Framework TFM in case they are currently only targeting .NET Standard, and B) turn on generation of binding redirects for the library project and pack the .dll.config with the NuGet package.

[KalleOlaviNiemitalo]

What would read the .dll.config if it were in the NuGet package? AFAIK, .NET Framework does not do so, unless the application creates an AppDomain with AppDomainSetup.ConfigurationFile pointing there. Does the .NET SDK have some feature that merges things from .dll.config files to the application config?

[joperezr]

You are right in that .NET Framework doesn't read the .dll.config by default, it would be up-to the host process loading the assemblies to do so. That said, most applications models will do so automatically (as the loader is the one in charge of lazy loading the assemblies), and so will most of the test runners (which usually Load the assembly into an appdomain but most do make sure that they read a config file if there is one) as well.

[jstedfast]

I've added <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects> to my .csproj but it doesn't seem to generate any *.dll.config files.

[KalleOlaviNiemitalo]

To generate binding redirects for a DLL, you'd have to set GenerateBindingRedirectsOutputType=true as well. That's the default for EXEs but not for DLLs.
https://github.com/dotnet/msbuild/blob/4c9e8aa8a981899b046beba9c290dbfc0a6d88ae/src/Tasks/Microsoft.Common.CurrentVersion.targets#L362

[KalleOlaviNiemitalo]

I don't believe that generating MimeKit.dll.config or MailKit.dll.config with binding redirects in it would fix the problem. Applications are more likely to reference those libraries directly than to load them in dedicated application domains.

Earlier, I filed dotnet/msbuild#7702 about a similar problem with System.Runtime.CompilerServices.Unsafe. That was resolved as a duplicate of #2547. The bug is that, if NuGet package includes both reference assemblies and runtime assemblies, then the runtime assemblies can have more dependencies than the reference assemblies (this is by design in Roslyn), but .NET SDK and MSBuild consider only the dependencies of the reference assemblies when they generate binding redirections. On net472, the System.Memory/4.5.1 package includes such a runtime-only dependency on System.Runtime.CompilerServices.Unsafe, hitting the bug. System.Memory/4.5.2 does not have this problem as the reference assemblies were removed in dotnet/corefx#33254.

However, MimeKit/3.4.3 already depends on System.Memory (>= 4.5.5), so this shouldn't be the problem.