Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[generator] Use + for nested types, not / #1302

Closed
wants to merge 3 commits into from

Conversation

jonpryor
Copy link
Member

@jonpryor jonpryor commented Feb 4, 2025

Context: dotnet/android#9747
Context: https://discord.com/channels/732297728826277939/732297837953679412/1336353039031734352
Context: https://discord.com/channels/732297728826277939/732297837953679412/1336358257769316372

The [Register] attribute provides "connector method" names, and for interface methods this will also include the name of the type which declares the method, which itself may be in a nested type:

namespace Android.App {
  public partial class Application {
    public partial interface IActivityLifecycleCallbacks : IJavaObject, IJavaPeerable {
      [Register (
          name: "onActivityCreated",
          signature: "(Landroid/app/Activity;Landroid/os/Bundle;)V",
          connector: "GetOnActivityCreated_Landroid_app_Activity_Landroid_os_Bundle_Handler:Android.App.Application/IActivityLifecycleCallbacksInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null")]
      void OnActivityCreated (Android.App.Activity activity, Android.OS.Bundle? savedInstanceState);

      // …
    }
  }
}

This output has been largely unchanged for years, but there is a problem with it: the connector parameter contains a nested type, and uses / to separate the "outer" type from the "inner" type.

This works on MonoVM.

This fails on NativeAOT and CoreCLR:

Could not resolve type 'Android.App.Application/IActivityLifecycleCallbacksInvoker' in assembly 'Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065'.

The / needs to be a + in order for Type.GetType() to find it:

[Register (
    name: "onActivityCreated",
    signature: "(Landroid/app/Activity;Landroid/os/Bundle;)V",
    connector: "GetOnActivityCreated_Landroid_app_Activity_Landroid_os_Bundle_Handler:Android.App.Application+IActivityLifecycleCallbacksInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null")]
void OnActivityCreated (Android.App.Activity activity, Android.OS.Bundle? savedInstanceState);

Update generator so that + is used within the connector parameter.

However, to maintain compatibility with existing binding assemblies, update Java.Interop.Tools.JavaCallableWrappers so that it replaces / with + on import, so that the resulting Java Callable Wrappers contain + and not /:

// Java Callable Wrapper
/* partial */ class MauiApplication_ActivityLifecycleCallbacks
{
  public static final String __md_methods;
  static {
    __md_methods =
      // …
      "n_onActivityCreated:(Landroid/app/Activity;Landroid/os/Bundle;)V:GetOnActivityCreated_Landroid_app_Activity_Landroid_os_Bundle_Handler:Android.App.Application+IActivityLifecycleCallbacksInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null\n" +
      // …
      "";
    mono.android.Runtime.register ("Microsoft.Maui.MauiApplication+ActivityLifecycleCallbacks, Microsoft.Maui", MauiApplication_ActivityLifecycleCallbacks.class, __md_methods);
  }
}

Context: dotnet/android#9747
Context: https://discord.com/channels/732297728826277939/732297837953679412/1336353039031734352
Context: https://discord.com/channels/732297728826277939/732297837953679412/1336358257769316372

The `[Register]` attribute provides "connector method" names, and for
interface methods this will also include the name of the type which
declares the method, which itself may be in a nested type:

	namespace Android.App {
	  public partial class Application {
	    public partial interface IActivityLifecycleCallbacks : IJavaObject, IJavaPeerable {
	      [Register (
	          name: "onActivityCreated",
	          signature: "(Landroid/app/Activity;Landroid/os/Bundle;)V",
	          connector: "GetOnActivityCreated_Landroid_app_Activity_Landroid_os_Bundle_Handler:Android.App.Application/IActivityLifecycleCallbacksInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null")]
	      void OnActivityCreated (Android.App.Activity activity, Android.OS.Bundle? savedInstanceState);

	      // …
	    }
	  }
	}

This output has been largely unchanged for *years*, but there is a
problem with it: the `connector` parameter contains a nested type,
and uses `/` to separate the "outer" type from the "inner" type.

This works on MonoVM.

This *fails* on NativeAOT and CoreCLR:

	Could not resolve type 'Android.App.Application/IActivityLifecycleCallbacksInvoker' in assembly 'Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065'.

The `/` needs to be a `+` in order for `Type.GetType()` to find it:

	[Register (
	    name: "onActivityCreated",
	    signature: "(Landroid/app/Activity;Landroid/os/Bundle;)V",
	    connector: "GetOnActivityCreated_Landroid_app_Activity_Landroid_os_Bundle_Handler:Android.App.Application+IActivityLifecycleCallbacksInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null")]
	void OnActivityCreated (Android.App.Activity activity, Android.OS.Bundle? savedInstanceState);

Update `generator` so that `+` is used within the `connector`
parameter.

However, to maintain compatibility with existing binding assemblies,
update `Java.Interop.Tools.JavaCallableWrappers` so that it replaces
`/` with `+` on import, so that the resulting Java Callable Wrappers
contain `+` and not `/`:

	// Java Callable Wrapper
	/* partial */ class MauiApplication_ActivityLifecycleCallbacks
	{
	  public static final String __md_methods;
	  static {
	    __md_methods =
	      // …
	      "n_onActivityCreated:(Landroid/app/Activity;Landroid/os/Bundle;)V:GetOnActivityCreated_Landroid_app_Activity_Landroid_os_Bundle_Handler:Android.App.Application+IActivityLifecycleCallbacksInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null\n" +
	      // …
	      "";
	    mono.android.Runtime.register ("Microsoft.Maui.MauiApplication+ActivityLifecycleCallbacks, Microsoft.Maui", MauiApplication_ActivityLifecycleCallbacks.class, __md_methods);
	  }
	}
@jonpryor
Copy link
Member Author

jonpryor commented Feb 4, 2025

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 5 out of 9 changed files in this pull request and generated no comments.

Files not reviewed (4)
  • tests/generator-Tests/Unit-Tests/CodeGeneratorExpectedResults/JavaInterop1/WriteNestedInterfaceTypes.txt: Language not supported
  • tests/generator-Tests/Unit-Tests/CodeGeneratorExpectedResults/XAJavaInterop1/WriteNestedInterfaceTypes.txt: Language not supported
  • tests/generator-Tests/Unit-Tests/DefaultInterfaceMethodsTests.cs: Evaluated as low risk
  • tests/generator-Tests/expected.xaji/AccessModifiers/Xamarin.Test.PublicClass.cs: Evaluated as low risk
jonpryor added a commit to dotnet/android that referenced this pull request Feb 4, 2025
Does It Build™?
jonpryor added a commit that referenced this pull request Feb 4, 2025
Context: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=10949922&view=logs&j=fc903661-9840-55b3-f380-f59f595bd021&t=96c38280-19e2-5308-b6a5-3c71c0cffc81
Context: #1302

PR #1302 is failing on Windows for a non-obvious reason:

    Java.Interop.Export-Tests -> D:\a\1\s\bin\TestRelease-net8.0\Java.Interop.Export-Tests.dll
      Possible reasons for this include:
        * You misspelled a built-in dotnet command.
        * You intended to execute a .NET program, but dotnet-D:\a\1\s\bin\Release-net8.0\/jcw-gen.dll does not exist.
      Could not execute because the specified command or file was not found.
        * You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH.
    ##[error]tests\Java.Interop.Export-Tests\Java.Interop.Export-Tests.targets(20,5): Error MSB3073: The command "dotnet "D:\a\1\s\bin\Release-net8.0\/jcw-gen.dll" "D:\a\1\s\bin\TestRelease-net8.0\Java.Interop.Export-Tests.dll" --codegen-target JavaInterop1 -o "obj\\Release-net8.0\/java" -L "D:\a\1\s\src\Java.Base\bin\Release\ref\." -L "D:\a\1\s\bin\Release-net8.0\ref\." -L "D:\a\1\s\bin\TestRelease-net8.0\ref\." -L "C:\hostedtoolcache\windows\dotnet\packs\Microsoft.NETCore.App.Ref\8.0.7\ref\net8.0\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.testplatform.testhost\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.codecoverage\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.linq.expressions\2.0.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.options\6.12.0.148\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\newtonsoft.json\13.0.1\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nuget.frameworks\5.11.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nunit\3.13.2\lib\netstandard2.0\." -L "D:\a\1\s\external\xamarin-android-tools\bin\Release\net6.0\ref\."" exited with code 1.
    D:\a\1\s\tests\Java.Interop.Export-Tests\Java.Interop.Export-Tests.targets(20,5): error MSB3073: The command "dotnet "D:\a\1\s\bin\Release-net8.0\/jcw-gen.dll" "D:\a\1\s\bin\TestRelease-net8.0\Java.Interop.Export-Tests.dll" --codegen-target JavaInterop1 -o "obj\\Release-net8.0\/java" -L "D:\a\1\s\src\Java.Base\bin\Release\ref\." -L "D:\a\1\s\bin\Release-net8.0\ref\." -L "D:\a\1\s\bin\TestRelease-net8.0\ref\." -L "C:\hostedtoolcache\windows\dotnet\packs\Microsoft.NETCore.App.Ref\8.0.7\ref\net8.0\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.testplatform.testhost\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.codecoverage\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.linq.expressions\2.0.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.options\6.12.0.148\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\newtonsoft.json\13.0.1\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nuget.frameworks\5.11.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nunit\3.13.2\lib\netstandard2.0\." -L "D:\a\1\s\external\xamarin-android-tools\bin\Release\net6.0\ref\."" exited with code 1. [D:\a\1\s\tests\Java.Interop.Export-Tests\Java.Interop.Export-Tests.csproj]

Given that *later* in the build, we see:

      jcw-gen -> D:\a\1\s\bin\Release-net8.0\jcw-gen.dll

it looks like we have a build ordering issue, and
`tests/Java.Interop.Export-Tests` is being run *before* `jcw-gen`?
@jonpryor jonpryor mentioned this pull request Feb 4, 2025
Copy link
Member

@jonathanpeppers jonathanpeppers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't follow what's happening in the CI build, but seems like these changes wouldn't cause it?

@jonpryor
Copy link
Member Author

jonpryor commented Feb 4, 2025

Indeed, I don't see how the changes in the PR would cause the build breakage on .NET Windows, and yet it does! #1303 builds without error.

jonpryor added a commit that referenced this pull request Feb 4, 2025
Context: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=10949922&view=logs&j=fc903661-9840-55b3-f380-f59f595bd021&t=96c38280-19e2-5308-b6a5-3c71c0cffc81
Context: #1302

PR #1302 is failing on Windows for a non-obvious reason:

    Java.Interop.Export-Tests -> D:\a\1\s\bin\TestRelease-net8.0\Java.Interop.Export-Tests.dll
      Possible reasons for this include:
        * You misspelled a built-in dotnet command.
        * You intended to execute a .NET program, but dotnet-D:\a\1\s\bin\Release-net8.0\/jcw-gen.dll does not exist. Could not execute because the specified command or file was not found. * You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH. ##[error]tests\Java.Interop.Export-Tests\Java.Interop.Export-Tests.targets(20,5): Error MSB3073: The command "dotnet "D:\a\1\s\bin\Release-net8.0\/jcw-gen.dll" "D:\a\1\s\bin\TestRelease-net8.0\Java.Interop.Export-Tests.dll" --codegen-target JavaInterop1 -o "obj\\Release-net8.0\/java" -L "D:\a\1\s\src\Java.Base\bin\Release\ref\." -L "D:\a\1\s\bin\Release-net8.0\ref\." -L "D:\a\1\s\bin\TestRelease-net8.0\ref\." -L "C:\hostedtoolcache\windows\dotnet\packs\Microsoft.NETCore.App.Ref\8.0.7\ref\net8.0\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.testplatform.testhost\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.codecoverage\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.linq.expressions\2.0.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.options\6.12.0.148\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\newtonsoft.json\13.0.1\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nuget.frameworks\5.11.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nunit\3.13.2\lib\netstandard2.0\." -L "D:\a\1\s\external\xamarin-android-tools\bin\Release\net6.0\ref\."" exited with code 1. D:\a\1\s\tests\Java.Interop.Export-Tests\Java.Interop.Export-Tests.targets(20,5): error MSB3073: The command "dotnet "D:\a\1\s\bin\Release-net8.0\/jcw-gen.dll" "D:\a\1\s\bin\TestRelease-net8.0\Java.Interop.Export-Tests.dll" --codegen-target JavaInterop1 -o "obj\\Release-net8.0\/java" -L "D:\a\1\s\src\Java.Base\bin\Release\ref\." -L "D:\a\1\s\bin\Release-net8.0\ref\." -L "D:\a\1\s\bin\TestRelease-net8.0\ref\." -L "C:\hostedtoolcache\windows\dotnet\packs\Microsoft.NETCore.App.Ref\8.0.7\ref\net8.0\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.testplatform.testhost\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.codecoverage\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.linq.expressions\2.0.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.options\6.12.0.148\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\newtonsoft.json\13.0.1\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nuget.frameworks\5.11.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nunit\3.13.2\lib\netstandard2.0\." -L "D:\a\1\s\external\xamarin-android-tools\bin\Release\net6.0\ref\."" exited with code 1. [D:\a\1\s\tests\Java.Interop.Export-Tests\Java.Interop.Export-Tests.csproj]

Given that *later* in the build, we see:

      jcw-gen -> D:\a\1\s\bin\Release-net8.0\jcw-gen.dll

it looks like we have a build ordering issue, and
`tests/Java.Interop.Export-Tests` is being run *before* `jcw-gen`?

Add a `@(PackagReference)` to `Java.Interop.Export-Tests.csproj`
to ensure `jcw-gen.csproj` is built first.
@jonpryor
Copy link
Member Author

jonpryor commented Feb 4, 2025

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

jonpryor added a commit that referenced this pull request Feb 4, 2025
Context: dotnet/android#9750

dotnet/android#9750 attempts to test #1302, and it
failed to build!

    …/src/Mono.Android/obj/Release/net10.0/android-35/mcw/Java.Util.Jar.Pack200.cs(229,54): error CS1109:
    Extension methods must be defined in a top level static class; IPackerExtensions is a nested class

Indeed, what we actually had was:

    abstract partial class Pack200 {
      public partial interface IPacker { /* … */ }

      static partial class IPackerExtensions {
        public static void Pack (this IPacker packer, Stream output, Stream input)
        {
          …
        }
      }
    }

`static` classes containing extension methods must be top-level,
and `IPackerExtensions` is instead nested within `Pack200`.

Fix `generator` so that this doesn't happen.
Context: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=10949922&view=logs&j=fc903661-9840-55b3-f380-f59f595bd021&t=96c38280-19e2-5308-b6a5-3c71c0cffc81
Context: #1302

PR #1302 is failing on Windows for a non-obvious reason:

    Java.Interop.Export-Tests -> D:\a\1\s\bin\TestRelease-net8.0\Java.Interop.Export-Tests.dll
      Possible reasons for this include:
        * You misspelled a built-in dotnet command.
        * You intended to execute a .NET program, but dotnet-D:\a\1\s\bin\Release-net8.0\/jcw-gen.dll does not exist. Could not execute because the specified command or file was not found. * You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH. ##[error]tests\Java.Interop.Export-Tests\Java.Interop.Export-Tests.targets(20,5): Error MSB3073: The command "dotnet "D:\a\1\s\bin\Release-net8.0\/jcw-gen.dll" "D:\a\1\s\bin\TestRelease-net8.0\Java.Interop.Export-Tests.dll" --codegen-target JavaInterop1 -o "obj\\Release-net8.0\/java" -L "D:\a\1\s\src\Java.Base\bin\Release\ref\." -L "D:\a\1\s\bin\Release-net8.0\ref\." -L "D:\a\1\s\bin\TestRelease-net8.0\ref\." -L "C:\hostedtoolcache\windows\dotnet\packs\Microsoft.NETCore.App.Ref\8.0.7\ref\net8.0\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.testplatform.testhost\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.codecoverage\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.linq.expressions\2.0.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.options\6.12.0.148\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\newtonsoft.json\13.0.1\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nuget.frameworks\5.11.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nunit\3.13.2\lib\netstandard2.0\." -L "D:\a\1\s\external\xamarin-android-tools\bin\Release\net6.0\ref\."" exited with code 1. D:\a\1\s\tests\Java.Interop.Export-Tests\Java.Interop.Export-Tests.targets(20,5): error MSB3073: The command "dotnet "D:\a\1\s\bin\Release-net8.0\/jcw-gen.dll" "D:\a\1\s\bin\TestRelease-net8.0\Java.Interop.Export-Tests.dll" --codegen-target JavaInterop1 -o "obj\\Release-net8.0\/java" -L "D:\a\1\s\src\Java.Base\bin\Release\ref\." -L "D:\a\1\s\bin\Release-net8.0\ref\." -L "D:\a\1\s\bin\TestRelease-net8.0\ref\." -L "C:\hostedtoolcache\windows\dotnet\packs\Microsoft.NETCore.App.Ref\8.0.7\ref\net8.0\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.testplatform.testhost\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.codecoverage\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.linq.expressions\2.0.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.options\6.12.0.148\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\newtonsoft.json\13.0.1\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nuget.frameworks\5.11.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nunit\3.13.2\lib\netstandard2.0\." -L "D:\a\1\s\external\xamarin-android-tools\bin\Release\net6.0\ref\."" exited with code 1. [D:\a\1\s\tests\Java.Interop.Export-Tests\Java.Interop.Export-Tests.csproj]

Given that *later* in the build, we see:

      jcw-gen -> D:\a\1\s\bin\Release-net8.0\jcw-gen.dll

it looks like we have a build ordering issue, and
`tests/Java.Interop.Export-Tests` is being run *before* `jcw-gen`?

Add a `@(PackagReference)` to `Java.Interop.Export-Tests.csproj`
to ensure `jcw-gen.csproj` is built first.
Context: dotnet/android#9750

dotnet/android#9750 attempts to test #1302, and it
failed to build!

    …/src/Mono.Android/obj/Release/net10.0/android-35/mcw/Java.Util.Jar.Pack200.cs(229,54): error CS1109:
    Extension methods must be defined in a top level static class; IPackerExtensions is a nested class

Indeed, what we actually had was:

    abstract partial class Pack200 {
      public partial interface IPacker { /* … */ }

      static partial class IPackerExtensions {
        public static void Pack (this IPacker packer, Stream output, Stream input)
        {
          …
        }
      }
    }

`static` classes containing extension methods must be top-level,
and `IPackerExtensions` is instead nested within `Pack200`.

Fix `generator` so that this doesn't happen.
@jonpryor
Copy link
Member Author

jonpryor commented Feb 4, 2025

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

jonpryor added a commit that referenced this pull request Feb 6, 2025
TODO: full explanation.

#1302 and dotnet/android#9750 has turned into a
bit of a boondoggle.  Maybe the better approach is to just update
`jcw-gen` to use `+` instead of `/` for nested types, instead of
*both* `jcw-gen` *and* `generator`, because the `generator` changes
just require changes *everywhere*.

Will this smaller fix work?
jonpryor added a commit that referenced this pull request Feb 6, 2025
Context: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=10949922&view=logs&j=fc903661-9840-55b3-f380-f59f595bd021&t=96c38280-19e2-5308-b6a5-3c71c0cffc81
Context: #1302

PR #1302 is failing on Windows for a non-obvious reason:

    Java.Interop.Export-Tests -> D:\a\1\s\bin\TestRelease-net8.0\Java.Interop.Export-Tests.dll
      Possible reasons for this include:
        * You misspelled a built-in dotnet command.
        * You intended to execute a .NET program, but dotnet-D:\a\1\s\bin\Release-net8.0\/jcw-gen.dll does not exist. Could not execute because the specified command or file was not found. * You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH. ##[error]tests\Java.Interop.Export-Tests\Java.Interop.Export-Tests.targets(20,5): Error MSB3073: The command "dotnet "D:\a\1\s\bin\Release-net8.0\/jcw-gen.dll" "D:\a\1\s\bin\TestRelease-net8.0\Java.Interop.Export-Tests.dll" --codegen-target JavaInterop1 -o "obj\\Release-net8.0\/java" -L "D:\a\1\s\src\Java.Base\bin\Release\ref\." -L "D:\a\1\s\bin\Release-net8.0\ref\." -L "D:\a\1\s\bin\TestRelease-net8.0\ref\." -L "C:\hostedtoolcache\windows\dotnet\packs\Microsoft.NETCore.App.Ref\8.0.7\ref\net8.0\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.testplatform.testhost\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.codecoverage\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.linq.expressions\2.0.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.options\6.12.0.148\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\newtonsoft.json\13.0.1\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nuget.frameworks\5.11.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nunit\3.13.2\lib\netstandard2.0\." -L "D:\a\1\s\external\xamarin-android-tools\bin\Release\net6.0\ref\."" exited with code 1. D:\a\1\s\tests\Java.Interop.Export-Tests\Java.Interop.Export-Tests.targets(20,5): error MSB3073: The command "dotnet "D:\a\1\s\bin\Release-net8.0\/jcw-gen.dll" "D:\a\1\s\bin\TestRelease-net8.0\Java.Interop.Export-Tests.dll" --codegen-target JavaInterop1 -o "obj\\Release-net8.0\/java" -L "D:\a\1\s\src\Java.Base\bin\Release\ref\." -L "D:\a\1\s\bin\Release-net8.0\ref\." -L "D:\a\1\s\bin\TestRelease-net8.0\ref\." -L "C:\hostedtoolcache\windows\dotnet\packs\Microsoft.NETCore.App.Ref\8.0.7\ref\net8.0\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.testplatform.testhost\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\microsoft.codecoverage\17.5.0-preview-20221003-04\lib\netcoreapp3.1\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.linq.expressions\2.0.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\mono.options\6.12.0.148\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\newtonsoft.json\13.0.1\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nuget.frameworks\5.11.0\lib\netstandard2.0\." -L "C:\Users\VssAdministrator\.nuget\packages\nunit\3.13.2\lib\netstandard2.0\." -L "D:\a\1\s\external\xamarin-android-tools\bin\Release\net6.0\ref\."" exited with code 1. [D:\a\1\s\tests\Java.Interop.Export-Tests\Java.Interop.Export-Tests.csproj]

Given that *later* in the build, we see:

      jcw-gen -> D:\a\1\s\bin\Release-net8.0\jcw-gen.dll

it looks like we have a build ordering issue, and
`tests/Java.Interop.Export-Tests` is being run *before* `jcw-gen`?

Add a `@(PackagReference)` to `Java.Interop.Export-Tests.csproj`
to ensure `jcw-gen.csproj` is built first.
@jonpryor
Copy link
Member Author

jonpryor commented Feb 6, 2025

This approach became an annoying boondoggle once I tried to integrate with Android: dotnet/android#9750

  • The changes to [Register] were (properly!) considered an API breaking change within Mono.Android.dll, which required updating acceptable-breakages-vReference-net10.0.txt to the tune of ~600 (!) entries
  • The changes to [Register] "broke" various pieces of linker-related code, such as MarshalMethodsClassififier and the linker itself. The fact that [Register] contained "IL-syntax" type names made it much easier to lookup types with Cecil. Attempting to change to Reflection syntax required finding all the places that performed such a lookup, which took a while (assuming I even found them all)
  • The changes to [Register] aren't required anyway! As initially mentioned, all existing binding assemblies are using IL syntax, so we have to fix them anyway!

#1304 just updates Java.Interop.Tools.JavaCallableWrappers so that generator output is unchanged. [Register] will continue to use IL nested type syntax, and That's Fine™. It results in a much smaller change to everything else.

@jonpryor jonpryor closed this Feb 6, 2025
jonpryor added a commit that referenced this pull request Feb 6, 2025
Context: dotnet/android#9747
Context: https://discord.com/channels/732297728826277939/732297837953679412/1336353039031734352
Context: https://discord.com/channels/732297728826277939/732297837953679412/1336358257769316372
Context: #1302
Context: dotnet/android#9750

The `[Register]` attribute provides "connector method" names, and for
interface methods this will also include the name of the type which
declares the method, which itself may be in a nested type:

	namespace Android.App {
	  public partial class Application {
	    public partial interface IActivityLifecycleCallbacks : IJavaObject, IJavaPeerable {
	      [Register (
	          name: "onActivityCreated",
	          signature: "(Landroid/app/Activity;Landroid/os/Bundle;)V",
	          connector: "GetOnActivityCreated_Landroid_app_Activity_Landroid_os_Bundle_Handler:Android.App.Application/IActivityLifecycleCallbacksInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null")]
	      void OnActivityCreated (Android.App.Activity activity, Android.OS.Bundle? savedInstanceState);

	      // …
	    }
	  }
	}

The `connector` value is copied as-is into Java Callable Wrappers,
as part of the `__md_methods` value and `Runtime.register()` call.
Given the C# type:

	partial class MauiApplication : Application {
	  partial class ActivityLifecycleCallbacks : Java.Lang.Object, Application.IActivityLifecycleCallbacks {
	    public void OnActivityCreated (Activity activity, Bundle? savedInstanceState) => …
	  }
	}

then `Java.Interop.Tools.JavaCallableWrappers` will produce:

	// Java Callable Wrapper
	/* partial */ class MauiApplication_ActivityLifecycleCallbacks
	{
	  public static final String __md_methods;
	  static {
	    __md_methods =
	      // …
	      "n_onActivityCreated:(Landroid/app/Activity;Landroid/os/Bundle;)V:GetOnActivityCreated_Landroid_app_Activity_Landroid_os_Bundle_Handler:Android.App.Application/IActivityLifecycleCallbacksInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null\n" +
	      // …
	      "";
	    mono.android.Runtime.register ("Microsoft.Maui.MauiApplication+ActivityLifecycleCallbacks, Microsoft.Maui", MauiApplication_ActivityLifecycleCallbacks.class, __md_methods);
	  }
	}

The `signature` and `connector` values from the `[Register(…)]` on
the method declaration are copied as-is into `__md_methods`.
As the `connector` value contains a `/`, `__md_methods` does as well.

This has worked fine for nearly 15+ years…on Mono/MonoVM.

This *fails* on NativeAOT and CoreCLR, as:

	Type.GetType ("Android.App.Application/IActivityLifecycleCallbacksInvoker, Mono.Android, …", throwOnError:true)

fails with:

	TypeLoadException: Could not resolve type 'Android.App.Application/IActivityLifecycleCallbacksInvoker' in assembly 'Mono.Android, …'.

The reason for the failure is that when using Reflection APIs such as
`Type.GetType()`, the [`Type.AssemblyQualifiedName` grammar][0] must
be followed, and that grammar uses `+` to "Precede a nested class",
*not* `/`.  (`/` isn't even in the Reflection grammar!)

(Aside: where does `/` come from then?  It's the *IL* separator for
nested types!)

For eventual CoreCLR and NativeAOT support, then, we need to replace
`/` with `+` *somewhere* before it hits `Type.GetType()`.

There are (at least?) three places to do so:

 1. Within `JniRuntime.JniTypeManager.RegisterNativeMembers()` or
    equivalent override.

 2. Within `generator`, updating the contents of `[Register]`.

 3. Within `Java.Interop.Tools.JavaCallableWrappers`.

(1) is rejected out of hand as it would be additional work done on-
device at runtime.  Why do that if we don't need to?

(2) was attempted in #1302 & dotnet/android#9750.
It turned into a bit of a boondoggle, because there are lots of
linker steps which interpret the `connector` value on `[Register]`,
and it "just worked" that the `connector` value contained IL names,
as the linker steps deal in IL!  Trying to update `connector` to
instead contain Reflection syntax required finding all the places in
the linker that used `connector` values, which was tedious & brittle.

"Just" (2) is also inadequate, as it would require new binding
assemblies to take advantage of, so (2) *also* needed (3).

Which brings us to (3), the current approach: *don't* alter the
semantics of the `connect` value within `[Register]`, and instead
require that `Java.Interop.Tools.JavaCallableWrappers` replace all
instances of `/` with `+` within `__md_methods`.  This is needed
*anyway*, for compatibility with existing bindings, and also avoids
lots of pain that (2) encountered.

With this approach, `generator` output is unchanged, and
`jcw-gen` output for `MauiApplication.ActivityLifecycleCallbacks`
becomes:

	// Java Callable Wrapper
	/* partial */ class MauiApplication_ActivityLifecycleCallbacks
	{
	  public static final String __md_methods;
	  static {
	    __md_methods =
	      // …
	      "n_onActivityCreated:(Landroid/app/Activity;Landroid/os/Bundle;)V:GetOnActivityCreated_Landroid_app_Activity_Landroid_os_Bundle_Handler:Android.App.Application+IActivityLifecycleCallbacksInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null\n" +
	      // …
	      "";
	    mono.android.Runtime.register ("Microsoft.Maui.MauiApplication+ActivityLifecycleCallbacks, Microsoft.Maui", MauiApplication_ActivityLifecycleCallbacks.class, __md_methods);
	  }
	}

[0]: https://learn.microsoft.com/en-us/dotnet/api/system.type.assemblyqualifiedname?view=net-9.0#remarks
jonpryor added a commit to dotnet/android that referenced this pull request Feb 20, 2025
dotnet/java-interop#1302 fixed the
`Could not determine Java type corresponding to System.Byte[]` crash.

What replaced it was:

    Mono.Android.NET_Tests, Java.InteropTests.JnienvTest.NewObjectArrayWithNonJavaType / Release
    System.ArgumentException : Could not determine Java type corresponding to System.Type, System.Private.CoreLib, Version=10.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e. Arg_ParamName_Name, type

The deal is that [`JavaNativeTypeManager.ToJniName(Type)`][0]
defaults to using `java/lang/Object` if there is no type mapping.

This is "reasonable" because of the "auto-box into
`Android.Runtime.JavaObject`" behavior, meaning it *is*
possible to create a Java array that "holds" `System.Type` instances.
(Because the Java-side array is actually a `java.lang.Object[]`, each
instance of which is an `Android.Runtime.JavaObject`, which in turn
holds the `System.Type` instance.)

However, this is *not* a semantic that
`JniRuntime.JniTypeManager.GetTypeSignature()` maintains, and thus
this was lost in 1b1f145.

Update `JNIEnv.FindClass(Type)` so that if `.GetTypeSignature()`
can't find the typemapping, default to `java/lang/Object`.

This allows all tests to pass, locally.

[0]: https://github.com/dotnet/java-interop/blob/62635a3ffee4c9ec421eb029b86e61267a544f92/src/Java.Interop.Tools.TypeNameMappings/Java.Interop.Tools.TypeNameMappings/JavaNativeTypeManager.cs#L164-L168
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants