plan a charm file resource built by local script

[addyess]

Applicable spec: KU-1646

Overview

Create/Handle integration tests plans with file resources

Rationale

Not all charms have image resources, some also use file resources. The workflows must grow the ability to plan, build, test, and publish with file resources.

Workflow Changes

The Plan, Build, Integration Test, and Publish workflows all get a bump to integrate with the new file based resources.

I didn't find any instructions on how to build the typescript so it was built with:

npm run format:write && npm run package

Checklist

• The contributing guide was applied
• The PR is tagged with appropriate label (urgent, trivial, complex)

I don't have the authority to set any of the tags

[addyess]

This branch cannot be merged until the changes to the uses: <path> is returned to its original settings under .github/workflows

[weiiwang01]

The pull request looks solid, thanks! Though I was wondering if we might explore an alternative approach for discovering the build script. I've added some details in the inline comments. Let me know your thoughts.

[weiiwang01]

Based on this code, should the argument name be --{resource-name}-resource?

            build.type === 'charm' ? 'charm-file' : `${name}-resource`

[weiiwang01]

I think maybe we should use multiple build scripts for different file resources. For example, if there are two file resources, foo and bar, we can create two build scripts, build-foo and build-bar, in the project directory. The operator-workflows should then run the respective build script to build the target resource.

This approach has the advantage that, currently, the source_file in the file resources build plan refers to the build target, whereas, in other build plans, it refers to the source file (charmcraft.yaml, rockcraft.yaml...) instead of the target file (charms, rocks). I assume this setup is required to pass the target file to the build-resource script as an argument. However, if we have one build script for each file resource, we can include respective build script in the build plan, making the file resources build plan more consistent with other types of artifacts.

[addyess]

right, i was hoping this line would generate exactly this discussion. I used build-resource.sh since i have seen this pattern in other charms which use file resources:

• calico
• flannel
• kubernetes-control-plane
• containerd

but in each of these above examples, it seems to closer resemble your description to have a build-{resource-name} executable.

I'm fine with this if we don't think it pollutes the working-directory with scripts.

I am still not 100% if I am fond of this solution as so many charms with file-resources may have a different resource based on the architecture or base. I have seen inconsistent ways of presenting charm file resource artifacts with respect to architecture.

[weiiwang01]

Yes, I understand what you mean. For example, in flannel, if we create a build script for each resource, we would have build-flannel-amd64, build-flannel-arm64, build-flannel-s390x, which is somewhat redundant.

In this scenario, make is actually a viable option in my opinion. You can create a makefile like the one below, and operator-workflows can simply call make {resource-name} to build the desired target, such as make flannel-arm64.

ARCHS := amd64 arm64 s390x

# build charm resources
.PHONY: $(foreach arch, $(ARCHS), flannel-$(arch))
$(foreach arch, $(ARCHS), flannel-$(arch)):
	@docker run --rm -e GOARCH=$(lastword $(subst -, , $@)) ...

but I do understand that make might not be the most ergonomic to use compared to a simple bash script, especially since, ideally, the make target should be the resource filename rather than the resource name. However, different resources can have the same filename; for example, flannel-amd64 and flannel-arm64 both have the filename flannel.tar.gz, so we can't use the filename as target.

[addyess]

@weiiwang01 ok, i went with the /build-{resource-name}.sh approach.

[weiiwang01]

Thanks, @addyess. If you'd like to preserve the original build-resource.sh method, I think we can do that as well. Just add a flag to the build plan structure so the build action knows which method to apply, whether it's build-resources.sh, build-{resource-name}.sh, or make {resource-name}. This way, we give developers maximum freedom to choose the method that best suits their project.

If you want to go with multiple build methods and keep the original build-resource.sh idea, I don’t think you need to implement all of them right away. Just keep the build-resource.sh method for now, and we can add more build methods in the future.

However, if you prefer using separate scripts for different resources, I have a trick to reduce duplication. We can create a master script, for example build-flannel, which switches to different builds based on the script name. For example:

#!/bin/bash

SCRIPT_NAME=$(basename "$0")
ARCH=${SCRIPT_NAME#build-flannel-}

if [[ "$ARCH" != "amd64" && "$ARCH" != "arm64" ]]; then
  echo "error: unknown architecture '$ARCH'."
  exit 1
fi

# build based on the ARCH
GOARCH="$ARCH" go build ./...

Then, we can create individual build scripts for each resource using symbolic links:

ln -s build-flannel-amd64 build-flannel
ln -s build-flannel-arm64 build-flannel

Calling each separate build script will build the specific resource:

./build-flannel-amd64 # builds the amd64 version
./build-flannel-arm64 # builds the arm64 version

[addyess]

i honestly thought about just expecting the charm metadata (via metadata.yaml or charmcraft.yaml) to provide some clue as to how to create this resource. At this moment. I think calling build-<resource>.sh from the working directory should be very clear. Our downstream projects can sort out how to make these scripts exist with symlinks or make files or whatever else.

[weiiwang01]

Sounds good to me, thanks!

[addyess]

@weiiwang01 any chance this can get an approval then?