arm64 support for Docker images

[jetxr]

Looks like the Docker images are built for amd64 only and don't support ARM architectures.

I was trying to run Sentry Relay on a AWS T4g instance, and ran into this error:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

Looks like the base image while running is debian:buster, which supports arm64. Any chance arm64 could be supported?

[flub]

Hi, does relay run fine on arm64 for you if you build it yourself? If so maybe you could consider making a PR to also build images for it.

[jjbayer]

@jetxr there is an effort to run the entire sentry stack on arm64: getsentry/self-hosted#914, but since we do not need it ourselves, there is no timeline for it.

[armenzg]

Currently there's no one working on it.

It seems that it might require some work to get the build going (at least building on my M1 is not working).

Once the build works, the image could be built as experimental on GH. We have an example image being built for Snuba.

docker build .

[+] Building 833.0s (19/24)
 => [internal] load build definition from Dockerfile                                                                                                       0.0s
 => => transferring dockerfile: 2.50kB                                                                                                                     0.0s
 => [internal] load .dockerignore                                                                                                                          0.0s
 => => transferring context: 348B                                                                                                                          0.0s
 => [internal] load metadata for docker.io/amd64/centos:7                                                                                                  1.8s
 => [internal] load metadata for docker.io/library/debian:buster-slim                                                                                      2.0s
 => [internal] load metadata for docker.io/getsentry/sentry-cli:1                                                                                          1.9s
 => [relay-deps 1/5] FROM docker.io/amd64/centos:7@sha256:dead07b4d8ed7e29e98de0f4504d87e8880d4347859d839686a31da35a3b532f                                11.9s
 => => resolve docker.io/amd64/centos:7@sha256:dead07b4d8ed7e29e98de0f4504d87e8880d4347859d839686a31da35a3b532f                                            0.0s
 => => sha256:dead07b4d8ed7e29e98de0f4504d87e8880d4347859d839686a31da35a3b532f 529B / 529B                                                                 0.0s
 => => sha256:eeb6ee3f44bd0b5103bb561b4c16bcb82328cfe5809ab675bb17ab3a16c517c9 2.75kB / 2.75kB                                                             0.0s
 => => sha256:2d473b07cdd5f0912cd6f1a703352c82b512407db6b05b43f2553732b55df3bc 76.10MB / 76.10MB                                                           7.3s
 => => extracting sha256:2d473b07cdd5f0912cd6f1a703352c82b512407db6b05b43f2553732b55df3bc                                                                  4.2s
 => [internal] load build context                                                                                                                          8.2s
 => => transferring context: 87.73MB                                                                                                                       8.2s
 => [sentry-cli 1/1] FROM docker.io/getsentry/sentry-cli:1@sha256:a4d754b52710bccf44656829611829966d3e00b79494d078e47fde15d876d647                         2.7s
 => => resolve docker.io/getsentry/sentry-cli:1@sha256:a4d754b52710bccf44656829611829966d3e00b79494d078e47fde15d876d647                                    0.0s
 => => sha256:861f6c12886e1d304a77b9c701861bd8935e4d7bba61e090b1b43cb97ef8458c 93B / 93B                                                                   0.4s
 => => sha256:a4d754b52710bccf44656829611829966d3e00b79494d078e47fde15d876d647 1.36kB / 1.36kB                                                             0.0s
 => => sha256:d9a6d31a5587649c5cc765359b0d6693ebb8818d4284f21a2befb7c23f3826d5 4.62kB / 4.62kB                                                             0.0s
 => => sha256:8663204ce13b2961da55026a2034abb9e5afaaccf6a9cfb44ad71406dcd07c7b 2.82MB / 2.82MB                                                             0.3s
 => => sha256:6b662ef96dc60ca1c40f49ddd55628b5b7d67e12b993e651c631589f4dff6662 271.61kB / 271.61kB                                                         0.5s
 => => extracting sha256:8663204ce13b2961da55026a2034abb9e5afaaccf6a9cfb44ad71406dcd07c7b                                                                  0.3s
 => => sha256:f40c19d5f8d514aeda81955df43efc3aa3792fca5eab8fb704a6359c0e5d8b0e 232B / 232B                                                                 0.5s
 => => sha256:3778e3073346d478ad6dfd8e33bf427a1092bdfeb53930b4867d915b8b20b663 6.99MB / 6.99MB                                                             1.4s
 => => extracting sha256:861f6c12886e1d304a77b9c701861bd8935e4d7bba61e090b1b43cb97ef8458c                                                                  0.0s
 => => extracting sha256:6b662ef96dc60ca1c40f49ddd55628b5b7d67e12b993e651c631589f4dff6662                                                                  0.1s
 => => extracting sha256:f40c19d5f8d514aeda81955df43efc3aa3792fca5eab8fb704a6359c0e5d8b0e                                                                  0.0s
 => => extracting sha256:3778e3073346d478ad6dfd8e33bf427a1092bdfeb53930b4867d915b8b20b663                                                                  0.8s
 => [stage-3 1/8] FROM docker.io/library/debian:buster-slim@sha256:fda76aa2ef4867e583dc8a7b86bbdb51118b8794c1b98aa4aeebaca3a1ad9c0f                        8.7s
 => => resolve docker.io/library/debian:buster-slim@sha256:fda76aa2ef4867e583dc8a7b86bbdb51118b8794c1b98aa4aeebaca3a1ad9c0f                                0.0s
 => => sha256:fda76aa2ef4867e583dc8a7b86bbdb51118b8794c1b98aa4aeebaca3a1ad9c0f 1.85kB / 1.85kB                                                             0.0s
 => => sha256:5bde28cdcc3d3c5bb4e8779f61d9c35f402611b7da3a34e1f17878fffff128c1 529B / 529B                                                                 0.0s
 => => sha256:0b7fab9b5413720fb59d1db49fd167b20cbb961d61947dfde19231b0f0914fc0 1.48kB / 1.48kB                                                             0.0s
 => => sha256:4931fb8ba4223cb35034141162105ee5482122692b2931eb69eec912ce64606d 25.91MB / 25.91MB                                                           4.5s
 => => extracting sha256:4931fb8ba4223cb35034141162105ee5482122692b2931eb69eec912ce64606d                                                                  3.9s
 => [stage-3 2/8] RUN apt-get update     && apt-get install -y ca-certificates gosu curl --no-install-recommends     && apt-get clean     && rm -rf /var  17.5s
 => [relay-deps 2/5] RUN yum -y update     && yum -y install centos-release-scl epel-release     && yum -y install cmake3 devtoolset-10 git     llvm-to  661.0s
 => [stage-3 3/8] RUN groupadd --system relay --gid 10001     && useradd --system --gid relay --uid 10001 relay                                            0.4s
 => [stage-3 4/8] RUN mkdir /work /etc/relay     && chown relay:relay /work /etc/relay                                                                     0.6s
 => [stage-3 5/8] WORKDIR /work                                                                                                                            0.1s
 => [relay-deps 3/5] RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal                                        36.8s
 => [relay-deps 4/5] COPY --from=sentry-cli /bin/sentry-cli /bin/sentry-cli                                                                                0.1s
 => [relay-deps 5/5] WORKDIR /work                                                                                                                         0.0s
 => [relay-builder 1/4] COPY . .                                                                                                                           0.3s
 => ERROR [relay-builder 2/4] RUN scl enable devtoolset-10 llvm-toolset-7.0 --     make build-linux-release     TARGET=x86_64-unknown-linux-gnu     REL  120.7s
------
 > [relay-builder 2/4] RUN scl enable devtoolset-10 llvm-toolset-7.0 --     make build-linux-release     TARGET=x86_64-unknown-linux-gnu     RELAY_FEATURES=ssl,processing,crash-handler:
#19 1.821 Submodule 'sentry-native' (https://github.com/getsentry/sentry-native.git) registered for path 'relay-crash/sentry-native'
#19 2.167 Submodule 'general/uap-core' (https://github.com/ua-parser/uap-core.git) registered for path 'relay-general/uap-core'
#19 2.713 Cloning into 'relay-crash/sentry-native'...
#19 6.295 Submodule path 'relay-crash/sentry-native': checked out '3436a29d839aa7437548be940ab62a85ca699635'
#19 6.814 Submodule 'external/breakpad' (https://github.com/getsentry/breakpad.git) registered for path 'external/breakpad'
#19 7.190 Submodule 'external/crashpad' (https://github.com/getsentry/crashpad.git) registered for path 'external/crashpad'
#19 7.576 Submodule 'external/libunwindstack-ndk' (https://github.com/getsentry/libunwindstack-ndk) registered for path 'external/libunwindstack-ndk'
#19 7.961 Submodule 'external/third_party/lss' (https://chromium.googlesource.com/linux-syscall-support) registered for path 'external/third_party/lss'
#19 8.554 Cloning into 'external/breakpad'...
#19 23.57 Submodule path 'relay-crash/sentry-native/external/breakpad': checked out '1f58de28fa974eac347e8bf3d998ba4465248cc8'
#19 24.32 Cloning into 'external/crashpad'...
#19 30.03 Submodule path 'relay-crash/sentry-native/external/crashpad': checked out 'e1e78cd7facbf554c6bbfb5dbbf58e864017bb99'
#19 30.59 Submodule 'third_party/lss/lss' (https://chromium.googlesource.com/linux-syscall-support) registered for path 'third_party/lss/lss'
#19 31.01 Submodule 'third_party/mini_chromium/mini_chromium' (https://chromium.googlesource.com/chromium/mini_chromium) registered for path 'third_party/mini_chromium/mini_chromium'
#19 31.41 Submodule 'third_party/zlib/zlib' (https://chromium.googlesource.com/chromium/src/third_party/zlib) registered for path 'third_party/zlib/zlib'
#19 32.03 Cloning into 'third_party/lss/lss'...
#19 34.92 Submodule path 'relay-crash/sentry-native/external/crashpad/third_party/lss/lss': checked out '7bde79cc274d06451bf65ae82c012a5d3e476b5a'
#19 35.63 Cloning into 'third_party/mini_chromium/mini_chromium'...
#19 38.18 Submodule path 'relay-crash/sentry-native/external/crashpad/third_party/mini_chromium/mini_chromium': checked out 'f9ae4322dfc5bcc14cc82d6005107f3b75ca2059'
#19 38.88 Cloning into 'third_party/zlib/zlib'...
#19 41.69 Submodule path 'relay-crash/sentry-native/external/crashpad/third_party/zlib/zlib': checked out '13dc246a58e4b72104d35f9b1809af95221ebda7'
#19 42.39 Cloning into 'external/libunwindstack-ndk'...
#19 45.00 Submodule path 'relay-crash/sentry-native/external/libunwindstack-ndk': checked out 'a4c27d48deff95fe922fe9733ef5c1339bdbf4fb'
#19 45.72 Cloning into 'external/third_party/lss'...
#19 48.10 Submodule path 'relay-crash/sentry-native/external/third_party/lss': checked out '171a36a8e0d1e456f63d342a09f811f9273a64af'
#19 48.79 Cloning into 'relay-general/uap-core'...
#19 58.42 Submodule path 'relay-general/uap-core': checked out 'bbd43aed9a623486191a33c3af9e463e89c85f7a'
#19 58.87 cd relay && cargo build --release --locked --features ssl,processing,crash-handler --target=x86_64-unknown-linux-gnu
#19 65.22     Updating crates.io index
#19 120.4 /bin/bash: line 1:  2680 Killed                  cargo build --release --locked --features ssl,processing,crash-handler --target=x86_64-unknown-linux-gnu
#19 120.5 make: *** [Makefile:27: build-linux-release] Error 137
------
executor failed running [/bin/sh -c scl enable devtoolset-10 llvm-toolset-7.0 --     make build-linux-release     TARGET=${BUILD_TARGET}     RELAY_FEATURES=${RELAY_FEATURES}]: exit code: 2

[jan-auer]

I recently looked into this. There are several points that need to be addressed:

1. Not all of Relay's features will build for arm64. Particularly, the crash-handler feature does not since it uses breakpad internally and that doesn't build on arm64. In short term, we can disable that feature for the arm64 build. In longer term, we should upgrade to an in-proc crash handler of sentry-native or even better port a full crash handler into sentry-rust, something that the Stacktrace team would have to look into.
2. Builds will be extremely slow until we start proper cross compiling. Cross compilation is definitely possible even for Relay's native dependencies (with the exception of breakpad), although it will require some work on the tooling.
3. Right now, there are two completely separate build pipelines. One runs release builds with a reduced feature set for "external" use directly on GitHub Actions hosts. The other one runs the "internal" docker container build on GCB. Ideally, we unify these into a build matrix, then simply copy the binary into a docker container. That would also allow us to customize dependency caching further.

[alekitto]

1. Not all of Relay's features will build for arm64. Particularly, the `crash-handler` feature does not since it uses _breakpad_ internally and that doesn't build on arm64. In short term, we can disable that feature for the arm64 build. In longer term, we should upgrade to an in-proc crash handler of `sentry-native` or even better port a full crash handler into `sentry-rust`, something that the Stacktrace team would have to look into.

Sorry, I have to correct you: breakpad DOES build on arm64, but not under CentOS 7 as its glibc version is too old.
Replacing it with a recent debian, it compiles successfully.

Here's a modified Dockerfile. I've built a multiplatform (amd64,arm64) image via buildx (cross-compiled) and successfully deployed it onto an ARM server.

For those who want to test it, I've published the relay image on dockerhub here

[jan-auer]

Does AWS T4g support that build with the updated glibc version? That was the primary reason why we switched to building on CentOS 7 in the first place.

[d8vjork]

+1 We need this as Graviton CPUs on AWS are cheaper and its the non-serverless part of the companies I'm working for.

Will be great to have this Relay at least built on ARM (docker and binaries) which I think the only problematic will be that Rust part anyway

[rissson]

We are also looking for this, for the reasons mentioned above. Perhaps @alekitto could open a PR for multiarch?

[MoogyG]

Same here

[iker-barriocanal]

I've tried getting the build working on #2370, it builds fine locally on an Intel Mac but I can't get it to work on Linux (CI on PR and local ubuntu VM). The error message during dependency installation is not helpful and I don't know what the root cause is, but I suspect there's a circular dependency with ca-certificates -> coreutils -> openssl-libs -> ca-certificates. I've run out of ideas, but happy to try new things if anyone has any suggestions. In the meantime, I'll move this back to our backlog.

[maximebeaudoin]

I don't understand why we couldn't have a docker image support for arm64 at this point. We really need this like really !

[Dav1dde]

@maximebeaudoin it's on our todo and something I also really want to get working, but unfortunately this does require a bigger restructure of the CI pipeline, so that's why it was moved back to the backlog.

[Dav1dde]

Took me longer than I wanted but we have ARM docker images now! We're still in the process of actually testing them fully (we still don't use them in production, yet). I don't expect any issues (Rust is very good in that regard) but any testing is very welcome.

[Damon99999]

1. Not all of Relay's features will build for arm64. Particularly, the `crash-handler` feature does not since it uses _breakpad_ internally and that doesn't build on arm64. In short term, we can disable that feature for the arm64 build. In longer term, we should upgrade to an in-proc crash handler of `sentry-native` or even better port a full crash handler into `sentry-rust`, something that the Stacktrace team would have to look into.

Sorry, I have to correct you: breakpad DOES build on arm64, but not under CentOS 7 as its glibc version is too old. Replacing it with a recent debian, it compiles successfully.

Here's a modified Dockerfile. I've built a multiplatform (amd64,arm64) image via buildx (cross-compiled) and successfully deployed it onto an ARM server.

For those who want to test it, I've published the relay image on dockerhub here

could you tell me how to use it,i'm new one on using docker,also my computer is M1 chip

[alekitto]

could you tell me how to use it,i'm new one on using docker,also my computer is M1 chip

Support for ARM architecture has been added to the official image, please refer to the official instructions on how to use it.