setup runc integration test

[saku3]

Description

This change allows runc integration tests to be executed using the youki runtime in CI.

.github/workflows/runc_integration_tests.yaml

This file configures the CI workflow to run the runc integration tests. It performs the following steps:

• Build youki
• Run runc integration tests:
  • Install necessary packages
  • Set up bats
  • Execute tests by running runc_integration_test.sh

scripts/runc_integration_test.sh

This script performs the following:

• Places the youki binary at the root of the runc repository, since bats expects the runtime binary to be in that location:
  https://github.com/opencontainers/runc/blob/main/tests/integration/helpers.bash#L15
• Runs tests using the bats command, targeting *.bats files under tests/integration:
  https://github.com/opencontainers/runc/tree/main/tests/integration
• To prevent test hangs in github actions, a mechanism was added to selectively skip individual test cases by injecting skip commands into .bats files:
  https://bats-core.readthedocs.io/en/stable/writing-tests.html#skip-easily-skip-tests

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update
• Refactoring
• Performance improvement
• Test updates
• CI/CD related changes
• Other:

Testing

• Added new unit tests
• Added new integration tests
• Ran existing test suite
• Tested manually (please provide steps)

I have confirmed that all tests pass with runc by executing the following command.(By specifying runc as the first argument, you can run the tests using the runc runtime.)

scripts/runc_integration_test.sh runc

Related Issues

Part of #3178

[saku3]

I might make further changes, but I’d appreciate any advice on whether the current direction is correct.

[utam0k]

It should be integrated with e2e.yaml.

[YJDoc2]

Hey so a couple of points here,

1. code-wise looks ok to me apart from utam0k's comment
2. I see in the run you have linked that there are lot of debug logs from Youki, even though in PR we are compiling youki in release mode, can you check that? We should not have debug logs in CI as it will clutter o/p.
3. For now, we don't need to run it on both arch, only amd64 with gnu should be enough. Once we have all failing tests triaged, and we are adding it as a standard CI, then we can use both archs.
4. I saw in the run you linked that all tests were run in 3-4 minutes. Are those all the runc tests or you ran just a subset/fail fast mode?

Thanks for your efforts :)

[saku3]

@utam0k @YJDoc2

Thank you for the review!

code-wise looks ok to me apart from utam0k's comment

I’ve updated it to run using e2e.yaml.

I see in the run you have linked that there are lot of debug logs from Youki, even though in PR we are compiling youki in release mode, can you check that? We should not have debug logs in CI as it will clutter o/p.

The debug logs are output as part of the tests, specifically when --debug is passed as an option to runc during execution.

For now, we don't need to run it on both arch, only amd64 with gnu should be enough. Once we have all failing tests triaged, and we are adding it as a standard CI, then we can use both archs.

I’ve updated the configuration accordingly.

I saw in the run you linked that all tests were run in 3-4 minutes. Are those all the runc tests or you ran just a subset/fail fast mode?

Currently, it's set up to run all of the tests (although a few known-hanging tests are skipped within runc_integration_test.sh).
However, the tests sometimes hang randomly during execution.
I'm not yet sure how best to handle those hangs.
In some runs, like the one below, all tests do complete:
https://github.com/saku3/youki/actions/runs/15792161456/job/44519468461?pr=13

I would appreciate any reviews or advice you could provide.

[utam0k]

How about passing a path of a binary for runtime?

[YJDoc2]

@saku3 , the CI seems to be failing. Or is it just a flake?

Also @utam0k , I think for now it is better if we put these tests in a separate CI job like podman and run it daily. Currently these are still failing, so maybe we will fix them first and then add them as a normal job in our CI. wdyt?

[saku3]

For now, I’ve set all failing tests to be skipped.
However, even with this change, the tests occasionally hang during execution. The reason for the hang is still under investigation.
It's still difficult to integrate this into the CI at this stage.

[YJDoc2]

Hey @saku3 , thanks for the changes. Let us do this - can we make the CI into a separate CI workflow like podman, I think similar to what you had done originally? That should be the only change from my side, and I'll merge the PR after that. Apologies it is taking so long from our side to get this finished 😓

[saku3]

@YJDoc2
Thank you for the review!
Sorry for the late response, but I’ve made the changes.

The updates since the last PR are as follows:

• As you pointed out, separated the tests, similar to the Podman CI workflow
• Prepared a file to list the test cases