Draft: Use a pipeline to load images into kind

[Blaimi]

This fixes the issue when kind runs in podman instead of docker

See: kubernetes-sigs/kind#2038
Fixes: #9901

Description
Because kind load docker-image doesn't work if kind is running with podman and the image to load is created with podman as well, the concept to load the image has to be changed. This problem is described in kubernetes-sigs/kind#2038. An "easy fix" for this is to save the image with the corresponding container engine as a docker-image to stdout and pass it to kind image-load /dev/stdin. By using localDocker like the builder does, podman can be selected by setting DOCKER_HOST to a podman-socket.

This implementation is more like a proof of concept than anything else. As everyone easily can see, I'm not a go-developer ;-)

This was tested as in "works on my machine". My machine at the moment:

• Fedora 34
• podman 5.6.2
• systemctls --user podman.socket is enabled and started
• DOCKER_HOST=unix:///var/run/user/$(id -u)/podman/podman.sock
• kind 0.30.0
• kubectl v1.28.15
• cluster version v1.34.0
• application used for testing: https://gitlab.com/biletado/backend/go/-/tree/feature/skaffold

User facing changes (remove if N/A)
If implemented correctly, nothing changes for docker users

Follow-up Work (remove if N/A)
Do a real implementation ;-)

[polarathene]

I may be mistaken, but care may want to be considered when dealing with large images (eg: 10GB+).

On WSL2 at least, I recall a similar issue with a tool wslcompact that performed the separate write/read operations for exporting a distro VM and then importing it for replacement (resolving a disk usage issue), and it did so via piping like proposed here?

That ended up requiring sufficient RAM if the VM disk image was large enough that it was problematic (it also involved implicit tar archive/unarchive of the VHDX file IIRC), which caused the operation to fail depending on the system memory available. Meanwhile manually doing the commands individually without a pipe was successful as actual disk was usable.

[Blaimi]

The implementation of kind load docker-image is actually using the same approach: https://github.com/kubernetes-sigs/kind/blob/91cc28782d7343dc1547c3d40e7136fd24ed5ddc/pkg/cmd/kind/load/docker-image/docker-image.go#L186

[polarathene]

The implementation of kind load docker-image is actually using the same approach

Sorry? I don't see piping there, just support for it. Here kind uses docker save to export .tar to a temp directory and then load that in which is not equivalent to piping.

I'm not that familiar with Go, so I may be mistaken but it looks like this PR is not writing the tar to disk, just piping the export into a reader. IIRC, if the operation doesn't block on the export itself completing and can instead maintain a smaller buffer during the operation to stream the export/import in smaller chunks, then memory overhead shouldn't balloon like I've observed with other tools like wslcompact.

---

Context

Import/export separately (required if using nerdctl for example, rather than implicitly supported by kind):

docker save localhost/example -o example.tar
kind load image-archive example.tar

What you proposed at kind to implement in Go via piping:

kind load image-archive -n kind-cluster <(podman save "$TAG" --format oci-archive)

That creates a temporary file descriptor, but I can't recall if that has that same implications that wslcompact has which does the following (PowerShell):

cmd /c "wsl --export ""$wsl_distro"" - | wsl --import wslcompact ""$tmp_folder"" -" 

Using an actual pipe operator | with stdout piped into stdin. AFAIK your use of process substitution <(...) is roughly equivalent with the file descriptor still being treated as an in-memory buffer by the kernel, nothing is written to disk.

I'm pretty sure that would run into the same piping issue, and so should the approach in Go then if the whole image is buffered into memory vs read from disk.

[Blaimi]

Podman desktop is also writing the image to the disk first: https://github.com/podman-desktop/podman-desktop/blob/main/extensions/kind/src/image-handler.ts#L80.

Maybe this is the saver approach, since two tools already are using this concept. Piping itself should not increase the memory usage, but I don't know, how this works, if you mix the systems where the image is coming from with the target system. E.g. if the image is loaded from docker (and therefore the docker wsl-vm) targeting a kind cluster running in podman (and therefore the podman wsl-vm)