`flux build artifact` doesn't follow symlinks

[attilaolah]

Describe the bug

When using flux build artifact --path=src --output=result.tgz, Flux seems to generate an empty OCI artifact if src contains symlinks to YAML manifest outside of src.

The use case is using Flux to generate OCI artifacts from a symlink tree built by Nix, with YAML files pointing to the read-only Nix store.

Steps to reproduce

mkdir ro rw
echo >ro/namespace.yaml 'apiVersion: v1
kind: Namespace
metadata:
  name = test'
chmod -w -R ro
ln -s ../ro/namespace.yaml rw/namespace.yaml
flux build artifact --path=rw --output=rw.tgz
tar tvf rw.tgz

Expected behavior

Expected output:

drwxr-xr-x 0/0               0 1970-01-01 01:00 .
-rw-r--r-- 0/0              56 1970-01-01 01:00 namespace.yaml

Actual output:

drwxr-xr-x 0/0               0 1970-01-01 01:00 .

Ideally Flux should resolve symlinks when creating OCI artifacts. Additionally, it should use a temporary directory (or allow specifying one) for any temporary files, which would allow working with source files located in the Nix store.

Screenshots and recordings

No response

OS / Distro

NixOS 24.05

Flux version

v2.4.0

Flux check

N/A

Git provider

N/A

Container Registry provider

GitHub (ghcr.io), but irrelevant here

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[makkes]

I'm not sure why symlinks are ignored but what I do know is that this behaviour has explicitly been implemented by @stefanprodan in https://github.com/fluxcd/pkg/pull/305/files#diff-5282f520c26e124aa9090bf1f79292fcc4aa23c4cecfc9e7df802d69c8e8189eR57. So maybe he has some more insights into the current behaviour.

[attilaolah]

Thanks for the pointer! For now I create a copy of the files which works, but it would be nice if I could somehow tell Flux to follow symlinks.

[stefanprodan]

Symlinks are skipped by the CLI since source-controller does not supports them and it will error out at pull time. I'm for adding a --resolve-symlinks flag to the CLI that would create a tmp dir with the whole content before generating the artifact.

[iceman91176]

Resolving symlinks should also happen with flux push artifact ... Copying around commonly used files work, but is not very DRY ;-)

[matheuscscp]

Same for flux diff artifact