Skip to content

Commit 4d5bfb9

Browse files
compnerdcompnerd
committed
Windows: add a toolchain image for 10.0.19044.1706 (21H2)
Add rules for building a docker image for the toolchain build at 10.0.19044.1706 (21H2). As the base image for the release is not available, use a slightly older release which should be compatible to avoid any issues due to kernel mismatches (the newer image will not run). This allows building the toolchain in a docker image with the latest stable release of Windows 10.
1 parent 594ab4b commit 4d5bfb9

File tree

5 files changed

+221
-0
lines changed

5 files changed

+221
-0
lines changed

‎swift-ci/README.md

+112
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,118 @@ docker run \
3535
/bin/bash -lc "cp -r /source/* /home/build-user/; ./swift/utils/build-script --preset buildbot_linux install_destdir=/home/build-user/swift-install installable_package=/home/build-user/swift-DEVELOPMENT-SNAPSHOT-$(date +'%F')-a.tar.gz"
3636
```
3737

38+
## Quick Start for Windows Development
39+
40+
The Windows Docker image will setup an enviornment with Python, Visual Studio
41+
Build Tools, and Git. It is setup to assume that the sources will be available
42+
in `S:\SourceCache`.
43+
44+
### Building and Tagging the image
45+
46+
Windows docker images must match the kernel version in the container and the
47+
host. You can identify the correct version by runing the `winver` command. The
48+
"OS Build" identifies the version suffix to apply to "10.0".
49+
50+
```powershell
51+
cd master\windows\10.0.19044.1706
52+
docker image build --compress -t swift:swiftci .
53+
```
54+
55+
### Running the image
56+
57+
```powershell
58+
docker run --rm -it -v %UserProfile%\data:S: swift:swiftci
59+
```
60+
61+
### Building the Toolchain
62+
63+
While we can build the toolchain in the containerized environment, the sources
64+
are expected to reside on the host and is passed into the docker container as a
65+
volume. The rest of automation expects that the Sources reside under a
66+
directory with the name `SourceCache`.
67+
68+
#### Clone the Sources
69+
70+
```cmd
71+
md %UserProfile%\data\SourceCache
72+
cd %UserProfile%\data\SourceCache
73+
74+
git clone -b stable/20211026 -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/llvm-project
75+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift
76+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-cmark cmark
77+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-experimental-string-processing
78+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-corelibs-libdispatch
79+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-corelibs-foundation
80+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-corelibs-xctest
81+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-argument-parser
82+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-crypto
83+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-driver
84+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-llbuild llbuild
85+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-package-manager
86+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-system
87+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-tools-support-core
88+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-installer-scripts
89+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/indexstore-db
90+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/sourcekit-lsp
91+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/jpsim/Yams
92+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/compnerd/swift-build
93+
git clone -b curl-7_77_0 -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/curl/curl
94+
git clone -b v2.9.12 -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/gnome/libxml2
95+
git clone -b v1.2.11 -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/madler/zlib
96+
git clone -b maint/maint-69 -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/unicode-org/icu
97+
```
98+
99+
#### Run Docker
100+
101+
```cmd
102+
docker run --rm -it -v %UserProfile%\data:S: swift:swiftci
103+
```
104+
105+
#### Build the Toolchain
106+
107+
This will build a full Swift toolchain distribution (llvm, clang, lld, lldb,
108+
swift, swift-package-manger, SourceKit-LSP) and the Windows SDK (x86, x64,
109+
ARM64).
110+
111+
```cmd
112+
S:
113+
S:\SourceCache\swift\utils\build.cmd
114+
```
115+
116+
#### Running Swift Tests
117+
118+
The toolchain tests require some modifications to the path to find some of the
119+
dependencies. The following will run the Swift test suite within the docker
120+
container:
121+
122+
```cmd
123+
path S:\b\1\bin;S:\b\1\tools\swift\libdispatch-windows-x86_64-prefix\bin;%Path%;%ProgramFiles%\Git\usr\bin
124+
ninja -C S:\b\1 check-swift
125+
```
126+
127+
#### Using the Toolchain
128+
129+
> **NOTE**: Running the test suite and using the toolchain near the production mode are mututally incompatible (due to the path changes).
130+
131+
The build will generate a toolchain image that is roughly similar to the
132+
installed version. The following can be run inside the docker container to use
133+
the toolchain:
134+
135+
```cmd
136+
set SDKROOT=S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk
137+
path S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\x64;S:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin;%Path%
138+
```
139+
140+
Because the toolchain is built in the volume which is backed by the host, the
141+
toolchain can be used on the host (assuming the dependencies such as Visual
142+
Studio is installed and the module modules deployed). The adjusted paths below
143+
should enable that:
144+
145+
```cmd
146+
set SDKROOT=%UserProfile%\data\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk
147+
path %UserProfile%\data\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\x64;%UserProfile%\data\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin;%Path%
148+
```
149+
38150
## Contributions
39151

40152
Contributions via pull requests are welcome and encouraged :)

‎swift-ci/master/windows/10.0.19044.1706/Dockerfile

+67
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# escape=`
2+
3+
# 10.0.19044.1706 is not yet published, use the 20H2 snapshot as that is
4+
# believed to work better than the newer image.
5+
FROM mcr.microsoft.com/windows/servercore:10.0.19042.1706 AS windows
6+
7+
LABEL maintainer="Swift Infrastructure <[email protected]>"
8+
LABEL description="Docker Container for the Swift programming language"
9+
10+
ARG GIT=https://github.com/git-for-windows/git/releases/download/v2.36.1.windows.1/Git-2.36.1-64-bit.exe
11+
ARG PYTHON=https://www.python.org/ftp/python/3.10.4/python-3.10.4-amd64.exe
12+
ARG WIX=https://wixtoolset.org/downloads/v3.14.0.6526/wix314.exe
13+
14+
# restore the default Windows shell for correct batch processing
15+
SHELL ["cmd", "/S", "/C"]
16+
17+
# Enable Developer Mode.
18+
RUN reg add "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\AppModelUnlock" /t REG_DWORD /f /v "AllowDevelopmentWithoutDevLicense" /d "1"
19+
20+
# Enable Long Paths
21+
RUN reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem" /t REG_DWORD /f /v "LongPathsEnabled" /d "1"
22+
23+
# Install Git.
24+
# See: git-[version]-[bit].exe /SAVEINF=git.inf and /?
25+
COPY git.inf .
26+
RUN `
27+
curl -SLo git.exe %GIT% `
28+
&& (start /w git.exe /SP- /VERYSILENT /SUPPRESSMSGBOXES /NOCANCEL /NORESTART /CLOSEAPPLICATIONS /FORCECLOSEAPPLICATIONS /LOADINF=git.inf ) `
29+
&& del /q git.exe git.inf
30+
31+
# Install Python.
32+
# See: https://docs.python.org/3.10/using/windows.html
33+
# FIXME: it appears that `PYTHONHOME` and `PYTHONPATH` are unset
34+
COPY unattend.xml .
35+
RUN `
36+
curl -SLo python.exe %PYTHON% `
37+
&& (start /w python.exe /quiet ) `
38+
&& del /q python.exe unattend.xml
39+
40+
# Install Visual Studio Build Tools
41+
RUN `
42+
curl -SLo vs_buildtools.exe https://aka.ms/vs/17/release/vs_buildtools.exe `
43+
&& (start /w vs_buildtools.exe --quiet --wait --norestart --nocache `
44+
--add Microsoft.VisualStudio.Component.Windows11SDK.22000 `
45+
--add Microsoft.VisualStudio.Component.VC.ATL `
46+
--add Microsoft.VisualStudio.Component.VC.ATL.ARM `
47+
--add Microsoft.VisualStudio.Component.VC.ATL.ARM64 `
48+
--add Microsoft.VisualStudio.Component.VC.CMake.Project `
49+
--add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 `
50+
--add Microsoft.VisualStudio.Component.VC.Tools.ARM64 `
51+
--add Microsoft.VisualStudio.Component.VC.Tools.ARM `
52+
|| IF "%EXITCODE%"=="3010" EXIT 0) `
53+
&& del /q vs_buildtools.exe
54+
COPY InstallModules.cmd .
55+
RUN InstallModules.cmd && del /q InstallModules.cmd
56+
57+
RUN `
58+
curl -SLo wix.exe %WIX% `
59+
&& (start /w wix.exe /install /quiet /norestart ) `
60+
&& del /q wix.exe
61+
62+
# FIXME: we should use a non-Administrator user
63+
# USER ContainerUser
64+
65+
ENV PYTHONUTF8=1
66+
# Default to powershell
67+
CMD ["powershell.exe", "-nologo", "-ExecutionPolicy", "Bypass"]

‎swift-ci/master/windows/10.0.19044.1706/InstallModules.cmd

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
@echo off
2+
setlocal
3+
set vswhere=%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe
4+
FOR /F "tokens=* usebackq" %%r IN (`"%vswhere%" -nologo -latest -all -prerelease -products * -property installationPath`) DO SET VsDevCmd=%%r\Common7\Tools\VsDevCmd.bat
5+
CALL "%VsDevCmd%" -no_logo -host_arch=amd64 -arch=amd64
6+
mklink "%UniversalCRTSdkDir%\Include\%UCRTVersion%\ucrt\module.modulemap" S:\SourceCache\swift\stdlib\public\Platform\ucrt.modulemap
7+
mklink "%UniversalCRTSdkDir%\Include\%UCRTVersion%\um\module.modulemap" S:\SourceCache\swift\stdlib\public\Platform\winsdk.modulemap
8+
mklink "%VCToolsInstallDir%\include\module.modulemap" S:\SourceCache\swift\stdlib\public\Platform\visualc.modulemap
9+
mklink "%VCToolsInstallDir%\include\visualc.apinotes" S:\SourceCache\swift\stdlib\public\Platform\visualc.apinotes
10+
endlocal
11+
@echo on

‎swift-ci/master/windows/10.0.19044.1706/git.inf

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
[Setup]
2+
NoIcons=1
3+
Components=gitlfs
4+
EditorOption=VIM
5+
PathOption=Cmd
6+
SSHOption=OpenSSH
7+
TurtoiseOption=false
8+
CURLOption=WinSSL
9+
BashTerminalOption=ConHost
10+
PerformanceTweaksFSCache=Enabled
11+
EnableSymlinks=Enabled
12+
EnablePseudoConsoltSupport=Disabled
13+
EnableFSMonitor=Enabled

‎swift-ci/master/windows/10.0.19044.1706/unattend.xml

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
<?xml version="1.0" encoding="utf-8"?>
2+
<Options>
3+
<Option Name="InstallAllUsers" Value="1" />
4+
<Option Name="AssociateFiles" Value="0" />
5+
<Option Name="PrependPath" Value="1" />
6+
<Option Name="Shortcuts" Value="0" />
7+
<Option Name="Include_doc" Value="0" />
8+
<Option Name="Include_debug" Value="0" />
9+
<Option Name="Include_dev" Value="1" />
10+
<Option Name="Include_exe" Value="1" />
11+
<Option Name="Include_launcher" Value="0" />
12+
<Option Name="InstallLauncherAllUsers" Value="0" />
13+
<Option Name="Include_lib" Value="1" />
14+
<Option Name="Include_symbols" Value="0" />
15+
<Option Name="Include_tcltk" Value="0" />
16+
<Option Name="Include_test" Value="0" />
17+
<Option Name="Include_tools" Value="0" />
18+
</Options>

0 commit comments

Comments
 (0)