Add E2E Tests

[zliang-akamai]

I am still working on the corresponding GHA workflow, but this can works locally when you have a kubeconfig file from a LKE.

Update:
GitHub workflow for this is actually quite complicated because we have to handle test image cleanup after the test, and there is not a official GitHub action or a simple docker command that can achieve this. We might have to invoke Docker Hub API directly for removing it. Would like to hear more thought on this.

hub-tool maybe an option to cleanup the images, but currently missing CI friendly login feature.

How to run

• export KUBECONFIG=/path/to/your/lke/kubeconfig
• export TEST_IMAGE_REPO=your-repo
• export TEST_IMAGE_TAG=your-tag
• go test ./e2e -count=1 -v

[nolancon]

@zliang-akamai just wondering if you've consider using Chainsaw as an e2e framework? I've used it on Provider Ceph and I believe it has been implemented in COSI Driver and CAPL recently too. Might be worth checking out if you haven't already. 😃

[zliang-akamai]

@nolancon thanks for the suggestion! I can take a look for sure.

[mhmxs]

Just FYI, in other projects we use Linode Cluster API because Terraform provider is deprecated as i know.

[zliang-akamai]

@nolancon @mhmxs, so decorator requires the Linode metadata API to work, so it won't just work in a local cluster, which means we need a real LKE cluster, which needs OpenTofu (TF) to deploy and cleanup. NVM, I think I misunderstood. So we use CAPI to provision a cluster in E2E of other projects? Is there any example of that?

If we will have to avoid using TF, another way is mocking both k8s and Linode Metadata APIs in some simple Go test cases, but it may be more like unit tests rather than e2e since no real APIs are used. Does it sound good to you guys?

[mhmxs]

Here is an example of CAPL management cluster: https://github.com/linode/linode-blockstorage-csi-driver/pull/163/files#diff-0908e8aad04dc51bce3a028b466879848a0cf5e57f2ee1af8bf08a599c600e23 and the one to create the E2E cluster on Linode: https://github.com/linode/linode-blockstorage-csi-driver/pull/163/files#diff-1bf5be570b019689c0945a7ad6bbbe07e8a05769685c47c07ec3a67a71d74d55

Mocking is only an option for unit tests. I did a short research, and i don't see any other option to properly test this project than how you did. But I strongly suggest to swap to CAPL and Chainsaw, because both would be used more widely in the future.

Here is the simplest adoption of chainsaw: https://github.com/linode/linode-cosi-driver/pull/55/files

If you have any question please feel free to reach me out.

[mhmxs]

For tooling management we use a mixed Makefile+Devbox combo, so it would be nice, if you can follow this style. (linked projects contain the examples)

[ChihweiLHBird]

@mhmxs Thank you! I will follow the style you mentioned. Mocking API is option one, and I recently learned about local (in-cluster) registry option, and I am going to do some research. Do you think a local image registry in the cluster can be the registry that the cluster pulls from?

[mhmxs]

Do you think a local image registry in the cluster can be the registry that the cluster pulls from?

@ChihweiLHBird The local registry has local url. So you have to

• pull every image
• retag them to local
• push to local
• edit every manifest before apply to replace images to local version
• (you have to ensure to cache images on github action, otherwise the full stuff doesn't make much sense if layers are not cached)

@nolancon ?