tests/ignition: Modify qemufailure to include bootfs test #3647

c4rt0 · 2023-10-18T11:52:33Z

openshift-ci · 2023-10-18T11:52:39Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

c4rt0 · 2023-10-18T12:03:47Z

The current state of things:

=== RUN   coreos.ignition.failure
=== RUN   coreos.ignition.failure/ignition
=== RUN   coreos.ignition.failure/bootfs
mke2fs 1.46.5 (30-Dec-2021)
Discarding device blocks: done                            
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: eff5788a-9ac9-468d-bcaf-36772cb07bb4
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

--- FAIL: coreos.ignition.failure (133.80s)
    --- PASS: coreos.ignition.failure/ignition (10.13s)
    --- FAIL: coreos.ignition.failure/bootfs (121.67s)
            qemufailure.go:57: timed out waiting for initramfs error: context deadline exceeded

mantle/kola/tests/ignition/qemufailure.go

c4rt0 · 2023-10-26T14:18:53Z

Currently getting an error:

=== RUN   coreos.unique.boot.failure
mke2fs 1.46.5 (30-Dec-2021)
Discarding device blocks: done                            
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: 96b87cc8-1dd9-4044-950e-e13567f18b4d
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

qemu-img: /var/tmp/mantle-qemu2849552875/disk2018417766: 'file' driver requires '/srv/tmp' to be a regular file
Could not open backing image.
--- FAIL: coreos.unique.boot.failure (2.02s)
        qemufailure.go:58: exit status 1
FAIL, output in _kola_temp/qemu-2023-10-26-1416-5229
Error: open _kola_temp/qemu-2023-10-26-1416-5229/properties.json: no such file or directory
2023-10-26T14:16:43Z cli: open _kola_temp/qemu-2023-10-26-1416-5229/properties.json: no such file or directory

c4rt0 · 2023-11-01T12:40:21Z

@jlebon Can you please revise the latest changes?

mantle/kola/tests/ignition/qemufailure.go

c4rt0 · 2023-11-07T13:40:52Z

Currently timing out:


[2023-11-07T12:08:27.253Z]         qemufailure.go:59: timed out waiting for initramfs error: context deadline exceeded

I've seen this behaviour here before when modifying BackingFile.

c4rt0 · 2023-11-08T14:52:52Z

/retest

mantle/kola/tests/ignition/qemufailure.go

c4rt0 · 2023-11-10T11:38:54Z

Updated the PR, this should verify that there is only one boot filesystem attached to the device. In order to make sure that the initramfs failure was related to the coreos-unique-boot.service I am checking the consolefile - curious to know if there is a better way to do it?
Appreciate an early review, but we (with @cverna) will start to work on the second test to use ignition config which creates a second boot file system.

jlebon

Some comments, but this is looking really good now. Nice work!

mantle/kola/tests/ignition/qemufailure.go

c4rt0

resolved

mantle/kola/tests/ignition/qemufailure.go

c4rt0 · 2023-11-22T12:12:21Z

/retest-required

c4rt0 · 2023-11-22T14:13:14Z

/retest-required

jlebon

This is looking close! coreos.ignition.failure is failing on RHCOS; possibly the error message is different there?

mantle/kola/tests/ignition/qemufailure.go

c4rt0 · 2023-11-23T14:28:25Z

This is looking close! coreos.ignition.failure is failing on RHCOS; possibly the error message is different there?

@jlebon - Thank you for all the comments and your time reviewing this issue!

mantle/kola/tests/ignition/qemufailure.go

See: coreos#2953 tests/ignition: Use ignition config to create a second bootfs Co-authored-by: JB Trystram <[email protected]> mantle/kola: Create functions to grep for patterns

c4rt0 · 2023-11-23T23:23:32Z

/retest

jbtrystram · 2023-11-27T14:40:07Z

When we were getting that working we ran into the issue where the test was passing, because it returned an error too early.
As the test returns an error, could we add a way to check the content to avoid false positive results ?

c4rt0 · 2023-11-27T16:01:19Z

When we were getting that working we ran into the issue where the test was passing, because it returned an error too early. As the test returns an error, could we add a way to check the content to avoid false positive results ?

I think during our meeting @jlebon referred to this comment

c4rt0 · 2023-11-27T16:07:09Z

Just for reference... changing the search string indeed fails the test as expected:

--- FAIL: coreos.unique.boot.failure (11.04s)
        qemufailure.go:69: pattern 'Error: System has 77 devices with a filesystem labeled 'booot'' in file '/var/tmp/mantle-qemu3317302958/console.log3137230128' not found
FAIL, output in _kola_temp/qemu-2023-11-27-1605-5449
Error: harness: test suite failed
2023-11-27T16:05:24Z cli: harness: test suite failed

jlebon

LGTM. Nice work!

jlebon · 2023-11-27T17:01:09Z

mantle/kola/tests/ignition/qemufailure.go

-			err = nil
+		resultingError := inst.WaitAll(ctx)
+		if resultingError == nil {
+			resultingError = fmt.Errorf("ignition unexpectedly succeeded")


I guess this error should probably be more generic now, since it's testing more than just Ignition failures. E.g. maybe just

initramfs unexpectedly succeeded

And the function name could be more specific, e.g. verifyInitramfsFailure().

But not worth a respin.

jlebon · 2023-11-27T17:04:09Z

/retest

c4rt0 · 2023-11-27T17:06:01Z

Awesome! Thanks @jlebon and @jbtrystram for all the help!

jlebon · 2023-11-28T18:08:29Z

Looks like we still have some issues to iron out. See #3669 and #3670.

c4rt0 · 2023-11-28T19:28:07Z

Looks like we still have some issues to iron out. See #3669 and #3670.

I will gladly look at both.

jbtrystram · 2023-11-30T10:28:51Z

/test all

openshift-ci bot added the do-not-merge/work-in-progress label Oct 18, 2023

c4rt0 force-pushed the add_kola_tests branch from c25f493 to 16d6714 Compare October 18, 2023 11:56

c4rt0 requested a review from jlebon October 18, 2023 11:57

jlebon reviewed Oct 18, 2023

View reviewed changes

mantle/kola/tests/ignition/qemufailure.go Outdated Show resolved Hide resolved

mantle/kola/tests/ignition/qemufailure.go Outdated Show resolved Hide resolved

mantle/kola/tests/ignition/qemufailure.go Show resolved Hide resolved

c4rt0 force-pushed the add_kola_tests branch from e2f503f to 0bc49aa Compare October 23, 2023 16:48

jlebon reviewed Oct 23, 2023

View reviewed changes

c4rt0 force-pushed the add_kola_tests branch from e6f7e4c to fca48b0 Compare October 25, 2023 22:42

c4rt0 force-pushed the add_kola_tests branch 2 times, most recently from 9b09876 to e919872 Compare October 31, 2023 18:40

c4rt0 self-assigned this Nov 1, 2023

c4rt0 marked this pull request as ready for review November 1, 2023 13:33

openshift-ci bot removed the do-not-merge/work-in-progress label Nov 1, 2023

c4rt0 force-pushed the add_kola_tests branch from 3b7ba9c to 5d672f6 Compare November 1, 2023 13:35

jlebon reviewed Nov 1, 2023

View reviewed changes

c4rt0 force-pushed the add_kola_tests branch from 5d672f6 to 572d6f9 Compare November 7, 2023 11:37

c4rt0 force-pushed the add_kola_tests branch from 572d6f9 to d10eb85 Compare November 8, 2023 14:51

jlebon reviewed Nov 9, 2023

View reviewed changes

c4rt0 force-pushed the add_kola_tests branch from d10eb85 to 370113e Compare November 10, 2023 11:33

c4rt0 force-pushed the add_kola_tests branch from bd33b17 to a5f703c Compare November 15, 2023 13:46

jlebon reviewed Nov 15, 2023

View reviewed changes

c4rt0 force-pushed the add_kola_tests branch 3 times, most recently from 9b89a62 to 236f749 Compare November 20, 2023 17:03

c4rt0 force-pushed the add_kola_tests branch 2 times, most recently from 7b43858 to 84602fa Compare November 22, 2023 10:05

c4rt0 commented Nov 22, 2023

View reviewed changes

jlebon reviewed Nov 22, 2023

View reviewed changes

c4rt0 force-pushed the add_kola_tests branch from 84602fa to c044a86 Compare November 23, 2023 09:46

jlebon reviewed Nov 23, 2023

View reviewed changes

c4rt0 force-pushed the add_kola_tests branch from c044a86 to a0f2359 Compare November 23, 2023 22:01

jlebon reviewed Nov 23, 2023

View reviewed changes

mantle/kola/tests/ignition/qemufailure.go Outdated Show resolved Hide resolved

tests/ignition: Modify qemufailure to include bootfs test

aa7c403

See: coreos#2953 tests/ignition: Use ignition config to create a second bootfs Co-authored-by: JB Trystram <[email protected]> mantle/kola: Create functions to grep for patterns

c4rt0 force-pushed the add_kola_tests branch from a0f2359 to aa7c403 Compare November 23, 2023 22:36

jlebon approved these changes Nov 27, 2023

View reviewed changes

jlebon enabled auto-merge (rebase) November 27, 2023 17:04

jlebon merged commit 0249856 into coreos:main Nov 27, 2023
2 checks passed

c4rt0 deleted the add_kola_tests branch November 27, 2023 19:41

c4rt0 mentioned this pull request Nov 27, 2023

Add kola tests for unique bootfs work #2953

Closed

marmijo mentioned this pull request Nov 28, 2023

coreos.unique.boot.failure kola test fails on aarch64 #3669

Closed

jlebon mentioned this pull request Nov 28, 2023

coreos.ignition.failure sometimes fails on RHCOS #3670

Closed

jlebon mentioned this pull request Nov 28, 2023

build(deps): bump github.com/coreos/stream-metadata-go from 0.4.3 to 0.4.4 #3668

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tests/ignition: Modify qemufailure to include bootfs test #3647

tests/ignition: Modify qemufailure to include bootfs test #3647

c4rt0 commented Oct 18, 2023

openshift-ci bot commented Oct 18, 2023

c4rt0 commented Oct 18, 2023

c4rt0 commented Oct 26, 2023 •

edited

Loading

c4rt0 commented Nov 1, 2023

c4rt0 commented Nov 7, 2023

c4rt0 commented Nov 8, 2023

c4rt0 commented Nov 10, 2023

jlebon left a comment

c4rt0 left a comment

c4rt0 commented Nov 22, 2023

c4rt0 commented Nov 22, 2023

jlebon left a comment

c4rt0 commented Nov 23, 2023 •

edited

Loading

c4rt0 commented Nov 23, 2023

jbtrystram commented Nov 27, 2023

c4rt0 commented Nov 27, 2023

c4rt0 commented Nov 27, 2023

jlebon left a comment

jlebon Nov 27, 2023

jlebon commented Nov 27, 2023

c4rt0 commented Nov 27, 2023

jlebon commented Nov 28, 2023

c4rt0 commented Nov 28, 2023

jbtrystram commented Nov 30, 2023

tests/ignition: Modify qemufailure to include bootfs test #3647

tests/ignition: Modify qemufailure to include bootfs test #3647

Conversation

c4rt0 commented Oct 18, 2023

openshift-ci bot commented Oct 18, 2023

c4rt0 commented Oct 18, 2023

c4rt0 commented Oct 26, 2023 • edited Loading

c4rt0 commented Nov 1, 2023

c4rt0 commented Nov 7, 2023

c4rt0 commented Nov 8, 2023

c4rt0 commented Nov 10, 2023

jlebon left a comment

Choose a reason for hiding this comment

c4rt0 left a comment

Choose a reason for hiding this comment

c4rt0 commented Nov 22, 2023

c4rt0 commented Nov 22, 2023

jlebon left a comment

Choose a reason for hiding this comment

c4rt0 commented Nov 23, 2023 • edited Loading

c4rt0 commented Nov 23, 2023

jbtrystram commented Nov 27, 2023

c4rt0 commented Nov 27, 2023

c4rt0 commented Nov 27, 2023

jlebon left a comment

Choose a reason for hiding this comment

jlebon Nov 27, 2023

Choose a reason for hiding this comment

jlebon commented Nov 27, 2023

c4rt0 commented Nov 27, 2023

jlebon commented Nov 28, 2023

c4rt0 commented Nov 28, 2023

jbtrystram commented Nov 30, 2023

c4rt0 commented Oct 26, 2023 •

edited

Loading

c4rt0 commented Nov 23, 2023 •

edited

Loading