Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bump CI VMs: local cache registry is now zstd #24280

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Bump CI VMs: local cache registry is now zstd #24280

wants to merge 5 commits into from
+469 −250

Conversation

edsantiago
Copy link
Member

The local cache registry now has images pushed with zstd:chunked
compression.

Built in: containers/automation_images#388

DO NOT MERGE yet. Am including #24238 because I depend on it

Signed-off-by: Ed Santiago [email protected]

None

@edsantiago edsantiago marked this pull request as draft October 15, 2024 16:59
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 15, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: edsantiago

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Oct 15, 2024
@packit-as-a-service Packit-as-a-Service
Copy link

Ephemeral COPR build failed. @containers/packit-build please check.

@edsantiago
Copy link
Member Author

Error: writing blob: archive/tar: write too long . I thought we had seen this before but I can't find it here nor c-s nor c-i.

Anyhow, just a quick reminder that zstd:chunked is not ready for prime time. Let's keep bashing away at it, but please let's not make it a default anywhere.

@Luap99
Copy link
Member

Luap99 commented Oct 15, 2024

@giuseppe ^^

archive/tar: write too long was seen quite a few times, I think it is more of a generic error
one with composefs containers/storage#1892

@edsantiago edsantiago mentioned this pull request Oct 15, 2024
Closed
@edsantiago
Copy link
Member Author

Testing now with

$ go mod edit --replace github.com/containers/storage=github.com/mtrmac/storage@chunked-size

@edsantiago edsantiago added the bloat_approved Approve a PR in which binary file size grows by over 50k label Oct 15, 2024
@edsantiago
Copy link
Member Author

Another new one, seen in APIv2. I can't reproduce:

           >$ ./bin/podman image build -t test:test -  [input is just 'FROM quay.io/libpod/testimage:20241011', that's all]
           >STEP 1/1: FROM quay.io/libpod/testimage:20241011
           >Trying to pull quay.io/libpod/testimage:20241011...
           >Getting image source signatures
           >Copying blob sha256:23620fb01e35328b4108dc2570057f67ec0f6cbfe4c392c07f70d46008fc312c
           >Copying blob sha256:25981b83f156d64ac55f45ea4c549fa54f4a27d20b4a9639a9efe965aed84617
           >Copying config sha256:b82e560ed57b77a897379e160371adcf1b000ca885e69c62cbec674777a83850
           >Writing manifest to image destination
           >COMMIT test:test
           >Error: copying image "f938cc09b1e10f9d4b7fed0463e47e1c358720faf4ee42f48bc8fc95e3d892a3": TOC digest "sha256:0bae53ca67c87e5996314b792bb75ea28c2109850554201ff8bfeeb805c8a5d7" for layer "54d51f339c073b4b9f6f70ba479185b24b4990fd344fd3b2a9edabea23667eff" is present but "expected-layer-diffid" flag is not a string

@edsantiago
Copy link
Member Author

"expected-diffid" bug showing up in int tests too

@edsantiago
Copy link
Member Author

New one

           $ podman [options] pull quay.io/libpod/cirros:latest
           Trying to pull quay.io/libpod/cirros:latest...
           Getting image source signatures
           Copying blob sha256:2266d94458b27e334b09df138fad51cb387fa331d70c27e0ee22ed4e3ae7e95e
           Copying blob sha256:bd001f579a0ca8301363741e82c8604f760ec62fe378f62cbdf37deb5526417b
           Copying blob sha256:98806be0b7ca57597019d042b5186ee5320f7957f0b322586024944221c5e9cc
           Error: copying system image from manifest list: reading blob sha256:98806be0b7ca57597019d042b5186ee5320f7957f0b322586024944221c5e9cc: not supported

edsantiago and others added 5 commits October 17, 2024 08:35
@edsantiago
Bump CI VMs: local cache registry is now zstd
a52dc63 
The local cache registry now has images pushed with zstd:chunked
compression.

Built in: containers/automation_images#388

Signed-off-by: Ed Santiago <[email protected]>
@edsantiago
go mod mtrmac/common@only-one-pull = 2202
1525b32 
Signed-off-by: Ed Santiago <[email protected]>
@edsantiago
mtrmac/image/v5@untrusted-digest-flag = 2603
efe783d 
Signed-off-by: Ed Santiago <[email protected]>
@edsantiago
go mod c/storage@main
fce557e 
Signed-off-by: Ed Santiago <[email protected]>
@mtrmac @edsantiago
FIXME Various zstd:chunked test changes
a1c21b1 
- Compare image configs, not IDs
- Try to actually handle additional store dependencies
- ??

Signed-off-by: Miloslav Trmač <[email protected]>
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 22, 2024
@openshift-merge-robot
Copy link
Collaborator

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bloat_approved Approve a PR in which binary file size grows by over 50k do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. release-note-none
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@edsantiago @Luap99 @openshift-merge-robot @mtrmac