Current arm64 6.6.79 (6.6) image is faulty

[florianl]

Minimal reproducer:

$ echo "FROM ghcr.io/cilium/ci-kernels:6.6.79" | docker buildx build --platform linux/arm64 --quiet --pull --output="./ci-kernels/arm64/6.6.79/" -
$ qemu-system-aarch64 -nographic  -append "console=ttyAMA0"  -M virt -cpu cortex-a57 -m 4G -kernel ./ci-kernels/arm64/6.6.79/boot/vmlinuz 

If one replace 6.6.79 with 6.6.41 in the above reproducer one can verify that the previous kernel image was working fine.
With the 6.6.79 image qemu-system-aarch64 does not even try to boot the kernel, as far as I can tell.

[lmb]

I've re-run the build: https://github.com/cilium/ci-kernels/actions/runs/13507765667 Does that fix it? Seems like sometimes the packaging goes wrong? No idea what it could be.

[florianl]

Looks like triggering a rebuild did not help. vmlinuz of 6.6.41 is of size 24M while 6.6.79 is still at 9.6M, so significantly smaller.

[lmb]

Does compiling on your machine reproduce as well? You should be able to use buildx.sh as outlined in the README.

[florianl]

I did run into multiple issues trying to build the ci-kernels locally - e.g. https://gist.github.com/florianl/2ff809fe4eb11d48e757821c7ae1ec98

Would you be open, to extend the CI and add a test to check the produced arm64 kernel? Something like this:

  test-arm64:
    name: Test ARM64 Kernel
    needs: build-and-push
    runs-on: ubuntu-24.04-arm
    if: ${{ inputs.build_arm64 == true }}
    
    permissions:
      contents: read
      packages: read
    
    steps:
    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v3
    
    - name: Set up Go
      uses: actions/setup-go@v4
      with:
        go-version: 'stable'
    
    - name: Install vimto and dependencies
      run: |
        sudo apt-get update
        sudo apt-get install -y --no-install-recommends qemu-system-aarch64
        CGO_ENABLED=0 go install lmb.io/vimto@latest
    
    - name: Fetch precompiled kernel
      run: |
        install -d ci-kernels
        echo "FROM ghcr.io/${{ github.repository_owner }}/ci-kernels:${{ inputs.kernel_version }}" \
          | docker buildx build --platform linux/arm64 \
            --quiet --pull --output="ci-kernels" -
    
    - name: Test kernel with vimto
      run: |
        kernel="$(pwd)/ci-kernels/boot/vmlinuz"
        echo "Testing kernel at $kernel"
        vimto -kernel "$kernel" exec -- /bin/true && echo "ARM64 kernel test successful"

[lmb]

Weird, I just tried it on my machine and it works. Kind of looks like a network connectivity issue?
I'd be happy to add a CI check, but I think the test has to happen before we push the image to the registry because that only happens on main builds. As far as I know this is kind of cumbersome to achieve in actions, because sharing the image between jobs is a pain.

[florianl]

On Fedora 40 I get the error from the linked gist. And on Debian 12 I do get the following error:

=> WARN: Requested platform "linux/amd64" does not match result platform "linux/arm64"                                                                                                                       0.0s
  1 warning found (use --debug to expand):
  - Requested platform "linux/amd64" does not match result platform "linux/arm64"

Time is limited atm so I can't debug docker buildx ....

Fair point - a CI test should happen before the image is pushed to the registry. And I agree, that handling docker builds in CI is tedious.