Skip to content

Switch autobuild OS to ARM runner #3470

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 13, 2025

Conversation

ann0see
Copy link
Member

@ann0see ann0see commented Jan 24, 2025

Short description of changes

Switches the build to ARM runners if an arm build is produced

CHANGELOG: Build: Switch Linux ARM builds to be built on arm runners

Context: Fixes an issue?

Fixes: #3466

Does this change need documentation? What needs to be documented and how?

No

Status of this Pull Request
Tested on debian arm64 on a raspberry pi. armhf and arm64 both run. I'd like to have an independent test too (best on raspbian).

What is missing until this pull request can be merged?

External testing

Checklist

  • I've verified that this Pull Request follows the general code principles
  • I tested my code and it does what I want
  • My code follows the style guide
  • I waited some time after this Pull Request was opened and all GitHub checks completed without errors.
  • I've filled all the content above

@ann0see ann0see force-pushed the autobuild/switchARM branch 3 times, most recently from f6fba38 to 5bcfb63 Compare January 24, 2025 20:44
@ann0see ann0see force-pushed the autobuild/switchARM branch from 5bcfb63 to cc78e4c Compare January 24, 2025 20:47
@ann0see ann0see requested a review from softins January 24, 2025 20:48
@ann0see ann0see added this to the Release 3.12.0 milestone Jan 24, 2025
@ann0see
Copy link
Member Author

ann0see commented Jan 24, 2025

ARM runners are considered public beta. I believe that if we don't remove the possibility to build on x86, it would be ok to merge this. But I'd also like to hear a general opinion.

Copy link
Member

@softins softins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't have time to test these builds for several days yet.

I've just looked at the build logs, and notice that native compilation only works for amd64 and arm64 architectures. Building armhf is still cross-compilation, but now on an arm64 runner. I assume that's expected.

@ann0see
Copy link
Member Author

ann0see commented Jan 24, 2025

Building armhf is still cross-compilation, but now on an arm64 runner. I assume that's expected.

Yes. That's expected.

@ann0see ann0see marked this pull request as ready for review January 24, 2025 22:31
@ann0see ann0see added the Linux Linux runtime issue label Feb 15, 2025
@ann0see ann0see requested review from softins and pljones March 22, 2025 19:37
Copy link
Member

@softins softins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Armhf build loads and runs ok on my Pi 4.

# Now take the original Ubuntu sources and limit those to the build host (i.e. non-TARGET_ARCH) architectures:
sudo sed -re 's/^deb /deb [arch=amd64,i386] /' -i /etc/apt/sources.list

if [[ "${APT_ARCH}" == "amd64" ]]; then
Copy link
Collaborator

@pljones pljones Mar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if "${APT_ARCH}" is arm64 and "${TARGET_ARCH}" is "amd64", for example? Should this explicitly error before this point unless"${APT_ARCH}" == "amd64"? (Rather than just not doing the copy.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a very good question. The present logic here seems to assume that cross-compilation is only done on an amd64 runner. That's no longer the case: we explicitly cross-compile from arm64 to armhf on an ARM runner. So I wonder whether this arch duplication is still needed in that case?

It would be good to get a reference for where this technique came from and why/when it's needed. Is @hoffie still around?

Copy link
Member

@softins softins Mar 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I think I understand it. It seems purely a method of specifying repo URLs independently for each architecture.

It seems most Ubuntu repos, such as the UK mirror only contain amd64 and i386, so the lines in /etc/apt/sources.list will only be able to find packages for those architectures.

When we want to cross-compile to non-Intel architectures, we need to be able to fetch packages from Ubuntu Ports instead. All the sed stuff is just setting up the routing for fetching those architectures, and also needs to prevent going there for amd64 and i386, which the Ports repo doesn't contain.

There is an example Gist here that shows a sources.list set up for cross-compilation in that way. The sed stuff in our repo just uses sources.list.d/ to organise the deb records for fetching non-native architectures.

I haven't had time yet to work out whether we need to do this for arm64 runners compiling for armhf. Maybe we don't.

Copy link
Collaborator

@pljones pljones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving on the basis that we'll find out if it breaks when it breaks.

@pljones pljones merged commit b15e8bd into jamulussoftware:main Apr 13, 2025
15 checks passed
@github-project-automation github-project-automation bot moved this from Waiting on Team to Done in Tracking Apr 13, 2025
@ann0see ann0see deleted the autobuild/switchARM branch April 14, 2025 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Linux Linux runtime issue
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Switch arm builds to ARM runners
3 participants