MavenReference Design Proposal

[wasabii]

It would be useful if users could include packages from Maven repositories within a project in a manner similar to PackageReferences.

I envision the feature being used similar to how PackageReference is within a MSBuild project file:

<ItemGroup>
  <MavenReference Identity="com.company.group/artifact" Version="1.0.0" />
</ItemGroup>

To consume, a user would have to add a PackageReference to a NuGet package we build that sets up the necessary targets. The package name could be something like IKVM.Build.Maven.

Behind the scenes, we would hook into the Restore pipeline. During restore, we search the configured Maven repositories for the proper artifact, and download it to Maven cache (~/.m2), much as NuGet downloads packages into the cache. We would likely consume the existing Maven libraries to accomplish this (we can!) These files would be downloaded the same place as regular Maven.

However, after that, we would run ikvmc on these artifacts, to generate .NET assemblies. These would be output into a similar cache as above, and care would have to be taken to ensure that we deal with versioning properly. The same user may reference Maven packages in projects that use a different version of IKVM (and thus ikvmc). Perhaps IKVM should grow an IkvmAssemblyVersion attribute of some kind, which we can decorate assemblies with, and increment each time a relevent change occurs. The cache folder can then be indexed by these values.

#54 May serve as the basis for this proposal, accomplishing the local JAR -> assembly conversion and maintenance, while this proposal simply fronts that.

The build system can then augment the Reference set pointing to these produced assemblies.

1. We would need to determine how to call the Maven resolver API.
2. Devise a layout for our compiled Jar cache.
3. Determine if virtual Reference values are suitable for MSBuild. What happens with version overlaps? Do we actually have to resolve the whole project graph as NuGet does?
4. Integration with Pack. Do we implicitly Private generated assemblies, or include a transitive reference to our MSBuild integration, so third parties consuming NuGet packages that reference Maven artifacts end up including the relevant targets?

[NightOwl888]

I just uploaded a proof-of-concept for how this could work, using only the mvn command line. It differs somewhat from what is described above, but it leverages the built-in dependency resolution logic of Maven. It also re-uses pom.xml configuration at the project level or higher or settings.xml at the user or machine level, so if the end user has a custom Maven package source and/or custom credentials, they can configure them using the Maven docs.

There are 3 stages to this task:

1. Resolve/download Maven Dependencies to obj/jar/ directory
2. Execute ikvmc on the items in obj/jar/ for each target framework
3. Self-copying the .targets file and <MavenReference>s recursively to each project that depends on us during dotnet pack

This only completes the first stage.

However, currently this is resolving dependencies at the project level only. It would be better to aggregate all of the <MavenReference>s for the entire solution so Maven can resolve any conflicts at that level, which would only become apparent after copying the binaries to the same folder as the application.

Per our discussion on Discord, re-packaging the IKVM converted assemblies is not in the cards (at least not for MavenRefernece). Instead, the .targets file copies itself like a virus when dotnet pack is executed on a consuming project. MSBuild doesn't execute .targets files for dependencies of dependencies, so we need to ensure that both the MavenRefernence processing and the actual references are copied up the chain each time someone consumes an assembly with Maven dependencies so they can be rebuilt in that project.

Maybe there is some way to tap into MSBuild at a lower level so we don't have to copy .targets files and <MavenReference>s like this, let me know if you know of a more elegant solution.

The second stage should be fairly straightforward at this point - run ikvmc for the <TargetFramework> and allow it to be overridden with <SetTargetFramework>TargetFramework=netcoreapp3.1</SetTargetFramwork> (which is how it is overridden on <PackageReference>). I haven't tried it, but I believe if the output of ikvmc is in the obj/$(Configuration)/$(TargetFramework) folder it will automatically be copied over to the bin/$(Configuration)/$(TargetFramework) folder. The project will also need some <Reference>s added so Intellisense and the build picks up the assemblies, I think. But these would need to be completely ignored during nuget pack.

Plus there are some more things to handle:

1. Strong naming
2. Adding the TargetFrameworkAttribute
3. Debug symbols

Unfortunately, the copy-references plugin for Maven doesn't function without XML configuration. So, the <MavenReference> elements are processed in order to generate a fake pom.xml file that contains the minimum configuration. If a project-level pom.xml exists, it is imported automatically at the obj/jar/ level. This should be recursive, but it isn't yet.

Anyway, I just wanted to put something together to get some feedback to see what is missing and whether there are more ideas that can be added to the mix. I think much of what is currently in IKVM.ProjectReferences.targets can eventually be put into a class library or two with some exposed MSBuild tasks as well as some APIs that the ikvm-cli tool can reuse.

[wasabii]

So this has been mostly completed in the ikvm-sdk-maven repository. As Maven is a third party thingy, I don't think it belongs in the ikvm repository. That said, here's how it works:

    <ItemGroup>
        <MavenReference Include="org.foo:barlib" Version="1.2.3" />
    </ItemGroup>

Yup that's it.

Each member of the MavenReference item group can either a) express the GroupId and ArtifactId in the identity, colon separated, or express it as metadata items instead.

    <ItemGroup>
        <MavenReference Include="randomname" GroupId="org.foo" ArtifactId="foolib" Version="1.2.3" />
    </ItemGroup>

MavenReferences are collected from three locations: MavenReference item group, ProjectReferences, and PackageReferences. All three of these lists of dependencies are merged together, and run through Maven's resolution logic, to produce a single set of JAR files to be compiled. This set is then emitted within the project as IkvmReference's. And the IkvmReference logic takes over from there.

Since MavenReferences are collected from ProjectReferences, this makes them transitive through projects. If project A depends on project B which depends on MavneReference A, then both Project B and Project A will end up attempting to resolve Maven Reference A. And each may in fact arrive at different conclusion (different version limitations, etc).

PackageReferences are already transitive. So, MavenReferences are collected by all PackageReferences of the current project (which includes those of child projects). Each PackageReferences is examined for a special file in it's nuget package. These files are TFM specific, and located like /maven/{tfm}/{PackageId}.pom.

So, the idea here is each NuGet package that depends on something in Maven declares that it depends on something in Maven in a POM file in itself.

The IkvmReference's produced by Maven reference are all set to Private. However, only COMPILE assets are set to ReferenceTargetAssembly true.

A project that contains non-transitive MavenReference items has a POM file produced in it's package output describing those. Thus completing the circle.

[wasabii]

This was completed in the ikvm-maven repository.