ctr-remote: add prefetch-list option

[wswsmao]

The optimize command requires a running container; however, this isn’t always feasible—for example, when the container needs GPU hardware while the service that performs image format conversion (e.g. a webhook) runs in an environment without GPUs. In such situations, users can collect the required dependency files themselves and provide them as input; the --prefetch-list option allows the format conversion to be carried out in the constrained environment.

[ktock]

A large portion of this function looks the same as a part of analyze function (

stargz-snapshotter/cmd/ctr-remote/commands/optimize.go

Lines 304 to 360 in ec299cc

	// Parse record file
	srcImg, err := is.Get(ctx, srcRef)
	if err != nil {
	return "", nil, nil, err
	}
	manifestDesc, err := containerdutil.ManifestDesc(ctx, cs, srcImg.Target, platforms.DefaultStrict())
	if err != nil {
	return "", nil, nil, err
	}
	p, err := content.ReadBlob(ctx, cs, manifestDesc)
	if err != nil {
	return "", nil, nil, err
	}
	var manifest ocispec.Manifest
	if err := json.Unmarshal(p, &manifest); err != nil {
	return "", nil, nil, err
	}
	// TODO: this should be indexed by layer "index" (not "digest")
	layerLogs := make(map[digest.Digest][]string, len(manifest.Layers))
	ra, err := cs.ReaderAt(ctx, ocispec.Descriptor{Digest: recordOut})
	if err != nil {
	return "", nil, nil, err
	}
	defer ra.Close()
	dec := json.NewDecoder(io.NewSectionReader(ra, 0, ra.Size()))
	added := make(map[digest.Digest]map[string]struct{}, len(manifest.Layers))
	for dec.More() {
	var e recorder.Entry
	if err := dec.Decode(&e); err != nil {
	return "", nil, nil, err
	}
	if *e.LayerIndex < len(manifest.Layers) &&
	e.ManifestDigest == manifestDesc.Digest.String() {
	dgst := manifest.Layers[*e.LayerIndex].Digest
	if added[dgst] == nil {
	added[dgst] = map[string]struct{}{}
	}
	if _, ok := added[dgst][e.Path]; !ok {
	added[dgst][e.Path] = struct{}{}
	layerLogs[dgst] = append(layerLogs[dgst], e.Path)
	}
	}
	}
	// Create a converter wrapper for skipping layer conversion. This skip occurs
	// if "reuse" option is specified, the source layer is already valid estargz
	// and no access occur to that layer.
	var excludes []digest.Digest
	layerOpts := make(map[digest.Digest][]estargz.Option, len(manifest.Layers))
	for _, desc := range manifest.Layers {
	if layerLog, ok := layerLogs[desc.Digest]; ok && len(layerLog) > 0 {
	layerOpts[desc.Digest] = []estargz.Option{estargz.WithPrioritizedFiles(layerLog)}
	} else if clicontext.Bool("reuse") && isReusableESGZLayer(ctx, desc, cs) {
	excludes = append(excludes, desc.Digest) // reuse layer without conversion
	}
	}
	return recordOut, layerOpts, excludeWrapper(excludes), nil

) except the manual creation of the record. So can you split this part to a common helper function so that it can be used for the both flags?

[wswsmao]

done

[ktock]

The usage of this flag and the file syntax should be documented somewhere such as docs/ctr-remote.md

[ktock]

Thanks