The type 'PartitionKeyInternal' is defined in an assembly that is not referenced.

[rjygraham]

Describe the bug
Receive the compiler error when trying to build netstandard 2.1 library referencing Azure.Cosmos version 4.0.0-preview3:

Error CS0012 The type 'PartitionKeyInternal' is defined in an assembly that is not referenced. You must add a reference to assembly 'Microsoft.Azure.Cosmos.Direct, Version=3.4.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.

To Reproduce

1. Create new netstandard 2.1 library
2. Add NuGet package reference to Azure.Cosmos version 4.0.0-preview3
3. Add code that references type Azure.Cosmos.PartitionKey
4. Attempt build of the project.

Expected behavior
Build completes successfully.

Actual behavior
Build fails with compiler error CS0012

Environment summary
SDK Version: 4.0.0-preview3
OS Version: Windows 10 19042.804
VS Version: Visual Studio Community 2019 16.8.4

Additional context
This issue is described in EFCore repo in the context of the v3 SDK:
dotnet/efcore#22231

The workaround proposed by @jongio dotnet/efcore#22231 (comment) fixes this issue in the 4.0.0-preview3 SDK but should not be necessary.

[ealsur]

Azure.Cosmos is a NET Standard 2.0 library, I'm assuming that means it can also be used on NET Standard 2.1? Not sure what are the rules for 2.1 and compatibility.

Could you provide a repro CSPROJ?

I created this CSPROJ:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Azure.Cosmos" Version="4.0.0-preview3" />
  </ItemGroup>

</Project>

Which contains just a single class that does this:

using Azure.Cosmos;
using System;

namespace Net21repro
{
    public class Class1
    {
        public Class1()
        {
            PartitionKey aPK = new PartitionKey("test");
        }
    }
}

And it builds:

I'm using NET 5.0.103:

The same goes on VS 2019:

[rjygraham]

Ok, this is a good one... It appears creating an instance of Azure.Cosmos.PartitionKey is not enough. The instance of Azure.Cosmos.PartitionKey must be in an async method:

using Azure.Cosmos;
using System.Threading.Tasks;

namespace CosmosRepro
{
	public class ReproClass
	{
		public async Task DoSomething()
		{
			var partitionKey = new PartitionKey("foo-bar-baz");
		}
	}
}

I verified the async keyword is the trigger and not Task. For example, the below code compiles fine:

using Azure.Cosmos;
using System.Threading.Tasks;

namespace CosmosRepro
{
	public class ReproClass
	{
		public Task DoSomething()
		{
			var partitionKey = new PartitionKey("foo-bar-baz");
			return Task.CompletedTask;
		}
	}
}

Additionally, my .csproj is pretty bare:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Azure.Cosmos" Version="4.0.0-preview3" />
  </ItemGroup>

</Project>

Let me know if you need any more details.

[ealsur]

I fear this might be related to .NET 5.

If you add a global.json file to the folder and force to use NET Core 3.1 for example, there is no problem:

With NET 5 as default

With a global.json file with the following content:

{
  "sdk": {
    "version": "3.1.200",
    "rollForward": "latestFeature"
  }
}

[ealsur]

What is also weird, is that if you remove the async keyword and return a Task and have another method consume that Task with await, there is no build issue:

        public Task DoSomething()
        {
            var partitionKey = new PartitionKey("foo-bar-baz");
            return Task.CompletedTask;
        }

        public async Task DoSomething2()
        {
            await DoSomething();
        }

But it's particularly related to having an async keyword in the method:

        public async Task DoSomething()
        {
            var partitionKey = new PartitionKey("foo-bar-baz");
            await Task.Delay(0);
        }

Again, this only happens on .NET 5, not NET Core 3.1.

I am no expert in the framework, but this really seems like an issue with async keywords in NET 5 and the way the Direct.DLL is included in the SDK nuget. I wouldn't know how to debug the building process or what is the difference.

[rjygraham]

Wondering if this should go to fx repo or Roslyn?

[jaredpar]

This behavior is "By Design" in the compiler.

The types in Azure.Cosmos.dll can’t be fully understood without looking at the Microsoft.Azure.Cosmos.Direct.dll reference which is not provided to the compiler. In particular in this case the compiler can't understand if PartitionKey is is an unmanaged or managed type without knowing more details about PartitionKeyInternal. That's due to it being an instance field inside of PartitionKey.

The reason this is happening only when the method is async is because the compiler handles captured locals differently based on whether they are managed or unmanaged. When exiting a method scope the compiler will zero out the captured locals, which are now fields of the state machine, in order to avoid holding onto unneeded references (directly or indirectly). There is no need to do this for unmanaged types though and hence we skip them. As mentioned though the compiler can't make this determination because it needs PartitionKey to make the determination and hence correctly issues an error.

There are other more direct ways to see this. For example if you try and use PartitionKey as a pointer, for a type parameter constrained to unmanaged, fixed statements, etc ...

This behavior changed in the 5.0 SDK because we fixed a bug. Previously we were suppressing this diagnostic and that caused us to incorrectly treat managed types as unmanaged in a few cases. Doing so opened up customers to more nefarious problems.

Overall this scenario has a more fundamental problem: it's not specifying the fully set of references needed to resolve the exposed types here. Specifically the compiler is not being provided Microsoft.Azure.Cosmos.Direct.dll. The compiler does as much as it can to still function when it's missing references. For example if you expose methods where you’re missing references for the types of the methods and you never call a method with that name then the compiler will generally be able to tolerate the missing reference. So if the uses cases are right and just the right set of references are missing it will work out.

The problem is the compiler makes no guarantees between versions though that we can tolerate the same set of missing references. Bug fixes, or even features can, force our hand here at considering defined members that we didn’t before and hence erroring because we can’t see all the types. In general my advice is to provide all the necessary references whenever possible.

[ealsur]

I believe this was fixed in V3 SDK (#1922) but never ported to V4 SDK (there were no new previews).

If we compare the Nuget package of V3 from before and after the fix:

Before:

After:

And if you instead of using Azure.Cosmos you use Microsoft.Azure.Cosmos, you don't face the issue.

[jaredpar]

That does look like it should fix it. If there isn't a ref folder then it will fall back to lib and that has all of the necessary references from what I can see.

[ealsur]

Also, if you try to use any V3 SDK previous to this fix, like 3.6.0, it throws the same error, but any SDK after the fix, like 3.16.0 does not.

[ealsur]

We don't have any new preview release slated to be released at this point, I'll update this thread if this can be fixed soon.

[matteobortolazzo]

Hi, do we have a fix for this?

Without PartitionKey methods like ReadItemAsync and DeleteItemAsync can't be used.

Or am I missing something?

Thanks

[bartelink]

You can reference the .Direct to remove the issue.

More importantly, you should consider using the actual supported library, which has two years of features and bugfixes included relative to Azure.Cosmos.

There will eventually be a released Azure.Cosmos that is Azure.Core API design compliant; atm there is not. (I also took an early dependency on Azure.Cosmos, but saw sense - there is no published timeline atm so the best thing to do is accept that fact given V4 does not offer any significant features over V3 as it stands -- over time, I'm sure it eventually will, and there will be cleanup given there's an opportunity to make some other breaking changes, but that's a future thing)

[matteobortolazzo]

Thanks for the clarification! I'll use V3 then 🙂