Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Download ILCompiler target pack when cross-compiling #30818

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Download ILCompiler target pack when cross-compiling #30818

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/Tasks/Microsoft.NET.Build.Tasks/ProcessFrameworkReferences.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -681,6 +681,14 @@ private bool AddToolPack(
targetIlcPack.SetMetadata(MetadataKeys.NuGetPackageId, targetIlcPackName);
targetIlcPack.SetMetadata(MetadataKeys.NuGetPackageVersion, packVersion);
TargetILCompilerPacks = new[] { targetIlcPack };

if (EnableRuntimePackDownload)
{
// We need to download the target runtime pack
TaskItem runtimePackToDownload = new TaskItem(targetIlcPackName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know why this condition is not needed for other platforms? -r osx-x64 on osx-arm64 or -r linux-arm64 on linux-x64 etc.

Copy link
Member Author

@filipnavara filipnavara Feb 23, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure it's needed for those as well. It's just that the original reporter happened to run things in order that populated the NuGet cache.

Weirdly I can confirm that works... will report back when I analyze the binlog.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's actually quite a mystery. It's not in _PackageToDownload but it's restored by the Restore task right after that. The only input that could be relevant is Microsoft.DotNet.ILCompiler dependency without a RID.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The difference is actually the dependency of the ILCompiler package.

This is the working one:

    "net8.0/osx-x64": {
      "Microsoft.DotNet.ILCompiler/8.0.0-preview.1.23110.8": {
        "type": "package",
        "dependencies": {
          "runtime.osx-x64.Microsoft.DotNet.ILCompiler": "8.0.0-preview.1.23110.8"
        },
        "build": {
          "build/Microsoft.DotNet.ILCompiler.props": {},
          "build/Microsoft.DotNet.ILCompiler.targets": {}
        }
      },

This is the non-working one:

    "net8.0/osx-arm64": {
      "Microsoft.DotNet.ILCompiler/8.0.0-preview.1.23110.8": {
        "type": "package",
        "build": {
          "build/Microsoft.DotNet.ILCompiler.props": {},
          "build/Microsoft.DotNet.ILCompiler.targets": {}
        }
      },

The runtime.json file in https://nuget.info/packages/Microsoft.DotNet.ILCompiler/8.0.0-preview.1.23110.8 doesn't have the osx-arm64 entry at all.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dotnet/runtime#82558

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weirdly I can confirm that works... will report back when I analyze the binlog.

Ah, same here, amd64 -> arm64 is restoring. I started testing it in a VM and made a Dockerfile 😅

# on linux host, make sure the environment is ready for cross-arch execution:
#    docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

FROM amd64/ubuntu:22.04

RUN apt update && apt install -y curl build-essential clang libicu70 file qemu-user-static binutils-aarch64-linux-gnu debootstrap qemu binfmt-support

RUN curl -SLO https://raw.githubusercontent.com/dotnet/arcade/main/eng/common/cross/build-rootfs.sh
RUN curl --create-dirs --output-dir arm64 -SLO https://raw.githubusercontent.com/dotnet/arcade/main/eng/common/cross/arm64/sources.list.jammy
RUN bash build-rootfs.sh arm64 jammy lldb14 --rootfsdir /crossrootfs/arm64

RUN mkdir ~/.dotnet && curl -sSL https://aka.ms/dotnet/8.0.1xx/daily/dotnet-sdk-linux-x64.tar.gz | tar xzf - -C ~/.dotnet

ENV PATH=${PATH}:/root/.dotnet

RUN dotnet new console -n app1
RUN echo '<configuration><packageSources><add key="dotnet8" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet8/nuget/v3/index.json" /></packageSources></configuration>' > nuget.config
RUN dotnet publish app1 -p:PublishAot=true -o dist -r linux-arm64 -p:SysRoot=/crossrootfs/arm64

ENTRYPOINT ["file", "dist/app1"]

docker build . -t test-arm64 followed by docker run test-arm64 prints:

dist/app1: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=19ec64d2622bbc890e741938f651d1473c0e9e50, for GNU/Linux 3.7.0, with debug_info, not stripped

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad to have secondary confirmation for what is happening. Microsoft.DotNet.ILCompiler has dependency for all the linux* RIDs on the RID-specific packages so it makes the restore working. osx-arm64 was the only outlier (but we are likely to hit this again with the ios-like platforms soon).

runtimePackToDownload.SetMetadata(MetadataKeys.Version, packVersion);
packagesToDownload.Add(runtimePackToDownload);
}
}
}
break;
Expand Down