Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing: Add integration testsuite #60

Merged
merged 17 commits into from
Nov 12, 2024
Merged

Conversation

SchoolGuy
Copy link
Member

@SchoolGuy SchoolGuy commented Oct 30, 2024

Fixes #51

This adds integration testing with the different versions of Cobbler. Workflow and scripts have been copied from the Terraform provider.

The goal of this PR is to rebuild the current Python testsuite that the backend has with the built-in CLI.

To make our code testable we need to be able to redirect the output to something that is not stdout. This can be achieved through cobras cmd.OutOrStdout() function.

Furthermore, we need to make generateCobblerClient() return an error and have that handled by cobras Command.RunE() method that we are using to execute our logic.

Lastly, we need to build ourselves a method to generate new instances of the commands because cmd.Execute() can only be called once.

@SchoolGuy SchoolGuy added this to the V0.0.1 milestone Oct 30, 2024
@SchoolGuy
Copy link
Member Author

Once GitHub Actions have recovered I will add more tests. Locally the one test I have added is failing successfully.

@SchoolGuy
Copy link
Member Author

Test setup in the CI with the different versions of Cobbler works as expected.

This is needed so during testing we can grab the things that would be printed to stdout and assert them.
This is so we can exit the CLI once a connection to Cobbler couldn't be successfully established.
@SchoolGuy SchoolGuy force-pushed the feature/add-integration-testsuite branch from 95f4873 to 6b40a6f Compare October 30, 2024 15:37
@SchoolGuy
Copy link
Member Author

The CLI is now prepared to be tested. I will start writing tests tomorrow.

@SchoolGuy SchoolGuy force-pushed the feature/add-integration-testsuite branch from 6b40a6f to e966aff Compare October 30, 2024 15:39
@SchoolGuy
Copy link
Member Author

We are affected of issue golang/go#33976 in our tests for the version command. The change is not released in any Go version at the moment. As such we need to test for the empty source in that test.

@SchoolGuy SchoolGuy force-pushed the feature/add-integration-testsuite branch 2 times, most recently from c87b7e2 to 2fe93f5 Compare October 31, 2024 15:45
@SchoolGuy
Copy link
Member Author

SchoolGuy commented Oct 31, 2024

Locally the only broken tests with 3.3.6 are related to Image and Systems. All individual tests of both item types fail with the same issues correspondingly. Furthermore, it seems that empty interfaces create XML issues with the system item type.

PR cobbler/cobblerclient#75 in the client will act as a fixup PR for locally fixed issues in the client.

@SchoolGuy
Copy link
Member Author

Image tests are locally passing now. The system tests are refusing to work because the sync tests in combination with the system add test is causing dhcpd to exit abnormally, thus causing an issue with the post change service restart trigger.

@SchoolGuy SchoolGuy force-pushed the feature/add-integration-testsuite branch from 892b378 to f918234 Compare October 31, 2024 19:20
@SchoolGuy
Copy link
Member Author

All tests are now passing locally. The issue was solved by adding a forced sleep for the duration of a second.

@SchoolGuy
Copy link
Member Author

SchoolGuy commented Oct 31, 2024

An update to 0.5.5 with bugfixes that are needed will cause the tests to pass.

Edit: The client version 0.5.5 will be released after cobbler/cobblerclient#75 was merged.

@SchoolGuy SchoolGuy force-pushed the feature/add-integration-testsuite branch from f918234 to 4b68df5 Compare November 4, 2024 09:18
@SchoolGuy SchoolGuy marked this pull request as ready for review November 4, 2024 09:19
@SchoolGuy SchoolGuy requested a review from a team November 4, 2024 09:19
@SchoolGuy
Copy link
Member Author

The Codacy Bash script errors are partly correct but I would need to fix that on the Terraform Provider as well. As none of those issues affect the delivered binary, I would like to skip those for now.

Copy link

@tiltingpenguin tiltingpenguin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The design of the tests looks sound to me, though for the future I would like to see some negative tests as well. But as long as the tests pass this is ok for me. I just found some nitpicks

testing/compose.yml Show resolved Hide resolved
cmd/system.go Outdated Show resolved Hide resolved
cmd/image_test.go Outdated Show resolved Hide resolved
@SchoolGuy SchoolGuy force-pushed the feature/add-integration-testsuite branch from 4b68df5 to 9b355c3 Compare November 12, 2024 10:11
@SchoolGuy SchoolGuy merged commit 4360345 into main Nov 12, 2024
17 checks passed
@SchoolGuy SchoolGuy deleted the feature/add-integration-testsuite branch November 12, 2024 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Add integration testsuite
2 participants