[XABT] Move JLO scanning needed for typemap generation to a linker step.

[jpobst]

Context: #9893
Context: #9930

This commit moves the process of scanning for JLOs needed for the typemap generation task to a new FindTypeMapObjectsStep "linker step".

The types are serialized to new <assembly>.typemap.xml files that sit beside the source assembly. A new file was used rather than adding more sections to the .jlo.xml file because this file contains a lot more types and will likely change more often on incremental builds, and we don't want to trigger ie: creating new Java stubs when it isn't needed.

This file is then read in the existing <GenerateTypeMappings> task to generate the final typemap file.

The existing method of JLO scanning for typemaps is still used when using LLVM marshal methods, as the marshal method rewriter runs after the linker steps, and when it changes the assemblies the MVID and type tokens no longer match the values saved in the .typemap.xml files.

Like #9893, this temporarily leaves the old typemap generation code in place, guarded behind the $(_AndroidJLOCheckedBuild) flag. This flag generates the typemap both the new and old way, and errors the build if there are differences.

Note that #10024 updated our "pipeline" to save assemblies after all steps have been run. However, looking for JLOs needed for typemap generation has to be run after assemblies are saved. This is because Release typemaps rely on assembly MVID and type tokens which change when the assemblies are modified.

Update the pipeline's "modified assembly saving" logic to itself be a pipeline step, so we can insert it before the new FindTypeMapObjectsStep step.

[jpobst]

A potential issue here is with cross-assembly duplicate detection.

Today, we are scanning all JLO derived types from all assemblies in a single loop, so duplicate detection logic like the following finds duplicates even if they are in different assemblies:

android/src/Xamarin.Android.Build.Tasks/Utilities/TypeMapCecilAdapter.cs

Lines 141 to 152 in 8c74126

	TypeMapDebugEntry oldEntry = duplicates [0];
	if ((td.IsAbstract || td.IsInterface) &&
	!oldEntry.TypeDefinition.IsAbstract &&
	!oldEntry.TypeDefinition.IsInterface &&
	td.IsAssignableFrom (oldEntry.TypeDefinition, cache)) {
	// We found the `Invoker` type *before* the declared type
	// Fix things up so the abstract type is first, and the `Invoker` is considered a duplicate.
	duplicates.Insert (0, entry);
	oldEntry.SkipInJavaToManaged = false;
	} else {
	// ¯\_(ツ)_/¯
	duplicates.Add (entry);

Because the new process only scans a single assembly at a time with Cecil, it will find duplicates in the same assembly, but not duplicates that may exist in other assemblies. An example is Java.Lang.Object:

Existing process:
- JavaName: "java/lang/Object" 
- ManagedName: "Java.Interop.JavaObject, Java.Interop" 
- SkipInJavaToManaged: "False" 
- DuplicateForJavaToManaged
  - JavaName: "java/lang/Object" 
  - ManagedName: "Java.Lang.Object, Mono.Android" 

New process:
- JavaName: "java/lang/Object" 
- ManagedName: "Java.Interop.JavaObject, Java.Interop" 
- SkipInJavaToManaged: "False" 
- DuplicateForJavaToManaged: null

It looks like we have 2 ordering semantics we have to preserve in replicating this logic:

• A declared type should take precedence over an "invoker" type.
• Types in Mono.Android should take precedence over other assemblies.

Aside from these 2 cases, which type is considered the "primary" and which type(s) are considered the "duplicates" seems to be luck of the draw.

[jonpryor]

@jpobst wrote:

type is considered the "primary" and which type(s) are considered the "duplicates" seems to be luck of the draw.

This is correct and incomplete. If you're only going by Java.Lang.Object.GetObject(handle), then you are correct.

However, most codepaths are not hitting Object.GetObject(IntPtr). Most codepaths are hitting Object.GetObject<T>(…) or Object.GetObject(…, Type targetType), which constrains which type that is used.

Thus, even if there are duplicate types (because bindings for a .jar are present in multiple assemblies), it generally won't matter because the APIs you're using will dictate the actual types used.

[jonathanpeppers]

Is the new trimmer step not setup for Release mode? I thought it would be listed here:

• https://github.com/dotnet/android/blob/main/src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.ILLink.targets

The PR description mentions it won't work in combination with $(AndroidEnableMarshalMethods)=true, but would it work for a Release build with marshal methods off?

[jpobst]

Is the new trimmer step not setup for Release mode?

Yes, it runs as part of the <AssemblyModifierPipeline> task, which is run by both the LinkAssembliesNoShrink and _AfterILLinkAdditionalSteps targets.

It works for Release mode when $(AndroidEnableMarshalMethods)=false.

[jonpryor]

As a "ha ha only serious" / "future coding guidelines" question/discussion: should we start using C# primary constructors?

public partial class FindTypeMapObjectStep (TaskLoggingHelper Log) : BaseStep, IAssemblyModifierPipelineStep {
    // …
}

[jpobst]

I don't love primary constructors, but if that's what we decide we want to standardize as a team it won't kill me. 😁

[grendello]

I'm not a fan of primary constructors either... :)

[jonpryor]

If we have a tuple return type, why not have out var foundJniNativeRegistration be a separate tuple value?

[jonpryor]

I wonder if this block and the "equivalent" block in the else branch should be moved to the end…

[jonpryor]

…to here, as:

if (!xml.HasDebugEntries && xml.ModuleReleaseData == null) {
    Log.LogDebugMessage ("No Java types found…");
    TypeMapObjectsXmlFile.WriteEmptyFile (destinationTypeMapXml, Log)
    return;
}

[jonpryor]

Should this method be responsible for checking for !this.HasDebugEntries && this.ModuleReleaseData == null and writing an empty file if there are no entries to write? This could simplify the call-sites.

Additionally, if we add a Log property to this class, the No Java types found/Wrote … message could also be moved here:

public void Export (string filename, string assemblyName)
{
    if (!HasDebugEntries && ModuleReleaseData == null) {
        Log.LogDebugMessage ($"No Java types found in '{assemblyName}'");
        TypeMapObjectsXmlFile.WriteEmptyFile (filename, Log)
        return;
    }
    Log.LogDebugMessage ($"Wrote '{filename}', {JavaToManagedDebugEntries.Count} JavaToManagedDebugEntries, {ManagedToJavaDebugEntries.Count} ManagedToJavaDebugEntries, FoundJniNativeRegistration: {FoundJniNativeRegistration}")
    // …
}

[jpobst]

Addressed feedback.

[Copilot]

Pull Request Overview

This PR refactors the typemap generation process by moving the JLO scanning to a new linker step and updating the overall assembly pipeline to better support incremental builds. Key changes include:

• Introducing a new FindTypeMapObjectsStep that writes typemap XML files beside source assemblies.
• Refactoring TypeMapGenerator and TypeMapCecilAdapter to use new adapter interfaces for handling debug and release typemap generation.
• Updating the AssemblyModifierPipeline to use void-based ProcessAssembly methods and adding a SaveChangedAssemblyStep.

Reviewed Changes

Copilot reviewed 11 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/Xamarin.Android.Build.Tasks/Utilities/TypeMapObjectsXmlFile.cs	Introduces XML export logic for typemap objects with support for both debug and release entries.
src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs	Refactors type mapping generation using new adapter abstractions and integrates the RunCheckedBuild flag.
src/Xamarin.Android.Build.Tasks/Utilities/TypeMapCecilAdapter.cs	Updates debug and release entry generation and duplicate handling logic.
src/Xamarin.Android.Build.Tasks/Tasks/GenerateTypeMappings.cs	Updates the task to generate typemaps from both managed and native states, with changes for safe file output accumulation.
src/Xamarin.Android.Build.Tasks/Tasks/AssemblyModifierPipeline.cs	Modifies pipeline steps to use void-based ProcessAssembly and introduces a new step to save changed assemblies.
src/Xamarin.Android.Build.Tasks/Linker/MonoDroid.Tuner/FindTypeMapObjectsStep.cs	Adds a new linker step that scans for Java types needed for typemap generation and writes them to an XML file.

Files not reviewed (2)

• src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.csproj: Language not supported
• src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets: Language not supported

[jonpryor]

Any particular reason that this is a nested type?

[jpobst]

Nope, unnested.

[jonpryor]

Context: ea399edb872588c16b124c4b9ab2232243b93016
Context: 787a2a6238942094f4a3fc62fb5595b46efa21ce
Context: b54ec05c4041ea2cb1e70058888a81e398df1e25
COntext: b11471bc574876e148ff40757c5d273fd6fbde9e

Move the process of scanning for `Java.Lang.Object` (JLO) and
`Java.Lang.Throwable` subclasses needed for the typemap generation
task to a new `FindTypeMapObjectsStep` "linker step".

The types are serialized to new `<assembly>.typemap.xml` files that
sit beside the source assembly.  A new file was used rather than adding
more sections to the `.jlo.xml` file because this file contains a lot
more types and will likely change more often on incremental builds, and
we don't want to trigger ie: creating new Java stubs when not needed.

This file is then read in the existing `<GenerateTypeMappings/>` task
to generate the final typemap file.

The existing method of JLO scanning for typemaps is still used when
using LLVM marshal methods, as the marshal method rewriter runs after
the linker steps, and when it changes the assemblies the MVID and type
tokens no longer match the values saved in the `.typemap.xml` files.

Like ea399edb, this temporarily leaves the old typemap generation code
in place, guarded behind the `$(_AndroidJLOCheckedBuild)` flag.
This flag generates the typemap both the new and old way, and errors
out if there are differences.

Note that b54ec05c updated our "pipeline" to save assemblies after
*all* steps have been run.  However, looking for JLOs needed for
typemap generation has to be run *after* assemblies are saved.
This is because Release typemaps rely on assembly MVID and type tokens
which change when the assemblies are modified.

Update the pipeline's "modified assembly saving" logic to itself be a
pipeline step, so we can insert it *before* the new
`FindTypeMapObjectsStep` step.

*Note*: previously, we scanned for all JLO subclasses from *all*
assemblies in a single loop, so [duplicate detection logic][0] finds
duplicates even if they are in different assemblies.

Consider `java.lang.Object`, which is bound as:

  * `Java.Lang.Object, Mono.Android`
  * `Java.Interop.JavaObject, Java.Interop`
  * *and also* several other unit test types as "aliases" (b11471bc)

When processing `Java.Interop.dll`, the `TypeMapDebugEntry` fields
would be:

  - JavaName: "java/lang/Object" 
  - ManagedName: "Java.Interop.JavaObject, Java.Interop" 
  - SkipInJavaToManaged: "False" 
  - DuplicateForJavaToManaged
    - JavaName: "java/lang/Object" 
    - ManagedName: "Java.Lang.Object, Mono.Android" 

In particular, `TypeMapDebugEntry.DuplicateForJavaToManaged` mentions
`Java.Lang.Object, Mono.Android` as the "main" typemap.

*Now* scanning is performed *per-assembly*; it will find duplicates in
the same assembly, but not duplicates that may exist in other
assemblies.  The `TypeMapDebugEntry` fields when processing
`Java.Interop.dll` are instead:

  - JavaName: "java/lang/Object" 
  - ManagedName: "Java.Interop.JavaObject, Java.Interop" 
  - SkipInJavaToManaged: "False" 
  - DuplicateForJavaToManaged: null

Note that `TypeMapDebugEntry.DuplicateForJavaToManaged` is null.


We have two ordering semantics we have to preserve in replicating
the original logic:

  - A declared type should take precedence over an "invoker" type.
  - Types in `Mono.Android` have precedence over other assemblies.

Aside from these two cases, which type is considered the "primary"
and which type(s) are considered the "duplicates" seems to be luck
of the draw.

Note that "luck" only enters the picture when using the non-generic
`JniRuntime.JniValueManager.GetPeer(JniObjectReference, Type?)` API
and providing `null` for the `Type?` parameter.  Whenever a `Type`
*is provided*, or a generic wrapper such as
`Java.Lang.Object.GetObject<T>()` is used, then value returned is
*constrained* to the specified type, reducing the likelihood of
obtaining an unexpected type.

[0]: https://github.com/dotnet/android/blob/8c741263550cc57bafc30193fa61f39b7df019bc/src/Xamarin.Android.Build.Tasks/Utilities/TypeMapCecilAdapter.cs#L141-L152