[native_toolchain_c] Support clang on Windows

[dcharkes]

We should add support for using Clang on Windows.

Using Clang on Windows will require passing in a "sysroot" from visual studio or cygwin for the system headers and system libraries to link against.

This will then inform what the native_assets_cli CCompilerConfig should look like to cover this use case.

• Remove CCompiler.{envScript,envScriptArgs} and replace by other means #1606

TODO:

• Support using clang if OS is Windows in package:native_toolchain_c.
• Set up a GitHub workflow to exercise this.
• (Optional) set up a new pkg bot on the Dart SDK that uses the clang for Windows.

[dcharkes]

The Clang for Windows on the Dart SDK seems to not have batteries included:

SEVERE: 2025-01-14 10:33:09.265940: lld-link: error: could not open 'libcmt.lib': no such file or directory
SEVERE: 2025-01-14 10:33:09.265940: lld-link: error: could not open 'oldnames.lib': no such file or directory
SEVERE: 2025-01-14 10:33:09.265940: clang: error: linker command failed with exit code 1 (use -v to see invocation)

logs

Searching the web for this error message mostly leads to reinstalls of toolchains or a reference to having to run vcvars.bat.

The Clang installed via the LLVM installer, or the Clang installed from MSVC have batteries included.

The clang pulled in the Dart SDK is a custom built one that maybe does not come with batteries included. Judging from https://github.com/dart-lang/sdk/blob/b5232ac632e26ffc4b4ca3d20afd51d337dbb39b/build/toolchain/win/BUILD.gn#L39-L58, we pull in the msvc parts via build/toolchain/win/msvc_toolchain.gni's msvc_toolchain.

Given that these errors do not occur on GitHub nor on my local installation, I think it's safe to say that the Clang-for-Windows on the Dart CI is the odd one out.

[mkustermann]

The Clang installed via the LLVM installer, or the Clang installed from MSVC have batteries included.

@dcharkes Are you sure about this? I would assume that Clang for Windows cannot re-distribute microsoft's artifacts (e.g. header files or libraries). Reading through clang manual seems to indicate that clang on windows will try to auto-detect it by looking up things in arguments (e.g. passed /winsysroot argument), environment and windows registry. So in your experiments how do you know that it was the clang-for-windows installation that had these things instead of it just being smart and figuring it out where they are located.

In any case, it does seem clang on windows does support some of the same things as msvc does (e.g. WindowsSdkDir environment vars).

In general we'd need the capability to allow cross-compiling, which reading e.g. this means we'd need

• toolchain - the cc/ar/link/... binaries
• sysroot (or equivalent) - the system libraries and header files
• target triplet - architecture, system/os, ...
• cpu/fpu/api - e.g. softfp or hardfp, sse3, neon, ...

The question is if we should

• explicitly model these things or
• implicitly (via general ccFlags, ldFlags, envVars, ...)

It may make sense to look into how cross compiling is done for Rust/Cargo.

[dcharkes]

So in your experiments how do you know that it was the clang-for-windows installation that had these things instead of it just being smart and figuring it out where they are located.

True, it might be smart and figuring it out by itself. Probably both my local machine and the GitHub bots have something in the registry. (It could also be the environment, but then it has to find it via one of the env vars that we allowlist.)

Let me try if I can spin up completely clean windows VM, only install LLVM Clang, and see if it fails to compile.

In general we'd need the capability to allow cross-compiling, which reading e.g. this means we'd need

• toolchain - the cc/ar/link/... binaries
• sysroot (or equivalent) - the system libraries and header files
• target triplet - architecture, system/os, ...
• cpu/fpu/api - e.g. softfp or hardfp, sse3, neon, ...

We have everything we need for all Dart and Flutter cross compilation targets at the moment.

• Toolchain is part of the config.
• Sysroot: The compilers support by Flutter work with the default sysroot (gcc/clang), or have a sysroot in a predictable location (Apple clang inside XCode). Except for MSVC for which we have the envvars.
• The target tripples and cpu/fpi/api are currently uniquely defined by OS+arch in Dart and Flutter. (And some of that is hardcoded into dart:ffi as well: for example Android Arm is always softfp and Linux Arm always hardfp.)

So currently, we only have the minimal info in the protocol, and construct things such as the target triple and the sysroot path for iOS/MacOS inside package:native_toolchain_c. However, most likely there will be embedders out there that target other combinations.

If we're talking about cpu/fpu/api, I'm not even sure if that should be in the CCompilerConfig, that should probably be in the CodeConfig?
Also, target-triple is defined as <arch><sub>-<vendor>-<sys>-<env>, where we already have CodeConfig.architecture.

It may make sense to look into how cross compiling is done for Rust/Cargo.

https://rust-lang.github.io/rustup/cross-compilation.html

It also uses target triples. @mosuem has been using this in icu4x: https://github.com/unicode-org/icu4x/blob/bfa1ddffd72783eb437f5a2ace25e165625020a9/ffi/dart/tool/build_libs.dart#L99 Also here CodeConfig (without CCompilerConfig) is enough to uniquely determine the target triples.

But indeed if we'd like to have an embedder to be able to target softfp-linux-arm, that softfp needs to be part of the CodeConfig, not the CCompilerConfig otherwise you wouldn't be able to select the right target for the rust compiler.

The question is if we should

• explicitly model these things or
• implicitly (via general ccFlags, ldFlags, envVars, ...)

I'd say things that can be consumed by other native compilers (Rust compiler) need to be explicitly modeled. The embedder doesn't know if a hook will use the provided C-toolchain from CCompilerConfig or if the hook will use something like the Rust compiler.

Another thing to consider is if a hook decides to be funky and uses multiple compilers. For example, if you use cgo you might want to pass the C sysroot to the Go compiler. For such use cases it's also nice if it's explicitly modeled (instead of fishing out the sysroot from a list of compiler arguments).

We can probably postpone adding sysroot/abi. But we need Windows "sysroot" now, so we need to decide how to model that. Maybe the vcvars.bat path should be called "sysroot" on Windows? I can't find any sysroot argument for the MSVC toolchain. But maybe that's more confusing that simplifying.

[mkustermann]

If we're talking about cpu/fpu/api, I'm not even sure if that should be in the CCompilerConfig, that should probably be in the CodeConfig?
Also, target-triple is defined as _{---, where we already have CodeConfig.architecture.}

Yes, that was meant as a general note.

The reason I mentioned softfp vs hardfp is that flutter engine (as well as Dart VM) actually allows building with hardfp or softfp (see tools/gn)

I'd say things that can be consumed by other native compilers (Rust compiler) need to be explicitly modeled. The embedder doesn't know if a hook will use the provided C-toolchain from CCompilerConfig or if the hook will use something like the Rust compiler.

Agreed! We only want the hook to produce a ELf/MachO/PE binary that is correctly built, whether it's with Rust or with C compiler. So it probably should be at CodeConfig and then it would be redundant to be also on CodeConfig.CompilerConfig.ccFlags/...

We can probably postpone adding sysroot/abi. But we need Windows "sysroot" now, so we need to decide how to model that. Maybe the vcvars.bat path should be called "sysroot" on Windows? I can't find any sysroot argument for the MSVC toolchain. But maybe that's more confusing that simplifying.

It may be that a CCompilerConfig.clang.sysroot is sufficient to configure clang on windows as well. If so, then the other mechanisms are msvc only and CCompilerConfig.msvc.* can encode them. (Only in the case clang-windows needs some of the msvc specific things, would the modelling of CCompilerConfig.clang fall apart)

[dcharkes]

The reason I mentioned softfp vs hardfp is that flutter engine (as well as Dart VM) actually allows building with hardfp or softfp (see tools/gn)

Yeah, but dart:ffi isn't supported (see runtime/vm/compiler/ffi/native_calling_convention.cc). So the CodeAssets will likely be useless until we get PRs to support that. 🙊

Agreed! We only want the hook to produce a ELf/MachO/PE binary that is correctly built, whether it's with Rust or with C compiler. So it probably should be at CodeConfig and then it would be redundant to be also on CodeConfig.CompilerConfig.ccFlags/...

Perfect, then we are on the same page!

It may be that a CCompilerConfig.clang.sysroot is sufficient to configure clang on windows as well. If so, then the other mechanisms are msvc only and CCompilerConfig.msvc.* can encode them. (Only in the case clang-windows needs some of the msvc specific things, would the modelling of CCompilerConfig.clang fall apart)

Wish me luck on another day of Windows VMs while I try to nail the behavior of Windows Clang down. 😅

[dcharkes]

Let me try if I can spin up completely clean windows VM, only install LLVM Clang, and see if it fails to compile.

clang: warning: unable to find a Visual Studio installation; try running Clang from a developer command prompt [-Wmsvc-not-found]
fatal error: 'stdio.h' file not found

The developer command prompt is the VsDevCmd.bat or vcvars.bat that sets up the environment vars. So it seems that Clang wants to be run with that.

Hardcoding an env-script (that is not registered in the Windows registry) indeed makes compilation with clang succeed:

  Future<Map<String, String>> resolveEnvironment(ToolInstance compiler) async {
    return await environmentFromBatchFile(
      Uri.file(
          r'C:\src\depot_tools\win_toolchain\vs_files\27370823e7\Windows Kits\10\bin/SetEnv.cmd'),
      arguments: ['/x86'],
    );

We can't test this on GitHub CI, it comes with MSVC pre-installed.