Skip to content

push: "Checksum Type mismatch occurred" when pushing to Wasabi (S3 compatible) #10695

Closed
@amarburg

Description

@amarburg

Bug Report

Description

Pushing individual files to our established bucket at Wasabi (S3-compatible cloud store), we get:

(S3 ACCESS and SECRET KEY defined in environment variables)
$ dvc push <push some_large_file>
Collecting                                                                          |0.00 [00:00,    ?entry/s]
ERROR: failed to transfer '4041c2812ed00efb5a0d57de2b8a9c4d' - [Errno 22] Checksum Type mismatch occurred, expected checksum Type: null, actual checksum Type: crc32: An error occurred (InvalidRequest) when calling the UploadPart operation: Checksum Type mismatch occurred, expected checksum Type: null, actual checksum Type: crc32
Pushing
ERROR: failed to push data to the cloud - 1 files failed to upload 

This is a bucket we have used consistently over the last ~2-3 years. We are able to pull from the bucket (i.e. it is not an acccess key issue)

Reproduce

  1. Set up S3 bucket at Wasabi.
  2. git init my_repo
  3. dvc init
  4. dvc remote add -d wasabi s3://bucket-name/
  5. dvc remote modify wasabi endpointurl https://s3.us-west-1.wasabisys.com
  6. dvc add big_file.txt
  7. AWS_ACCESS_KEY_ID="user" AWS_SECRET_ACCESS_KEY="secret" dvc push -r wasabi big_file.txt

Expected

File should be pushed to S3 remote and available for pulling by other users.

Environment information

Tested on both Ubuntu 24.04 and 20.04. DVC from snap:

$ snap info dvc
name:      dvc
summary:   Data Version Control
publisher: Casper (casper-dcl)
store-url: https://snapcraft.io/dvc
contact:   [email protected]
license:   Apache-2.0
description: |
  Git for Data & Models https://dvc.org
commands:
  - dvc
snap-id:      ceYKZQ2pf75cN9OVM33Bk36vVEwz3HaP
tracking:     v2/stable
refresh-date: yesterday at 13:58 PST
channels:
  latest/stable:    3.59.1  2025-02-16 (1488) 404MB classic
...                            
  v2/stable:        3.59.1  2025-02-16 (1488) 404MB classic
$ dvc doctor
DVC version: 3.59.1 (snap)
--------------------------
Platform: Python 3.12.9 on Linux-6.8.0-54-generic-x86_64-with-glibc2.31
Subprojects:
        dvc_data = 3.16.9
        dvc_objects = 5.1.0
        dvc_render = 1.0.2
        dvc_task = 0.40.2
        scmrepo = 3.3.10
Supports:
        azure (adlfs = 2024.12.0, knack = 0.12.0, azure-identity = 1.20.0),
        gdrive (pydrive2 = 1.21.3),
        gs (gcsfs = 2025.2.0),
        hdfs (fsspec = 2025.2.0, pyarrow = 19.0.0),
        http (aiohttp = 3.11.12, aiohttp-retry = 2.9.1),
        https (aiohttp = 3.11.12, aiohttp-retry = 2.9.1),
        oss (ossfs = 2023.12.0),
        s3 (s3fs = 2025.2.0, boto3 = 1.36.3),
        ssh (sshfs = 2025.2.0),
        webdav (webdav4 = 0.10.0),
        webdavs (webdav4 = 0.10.0),
        webhdfs (fsspec = 2025.2.0)
Config:
        Global: /home/aaron/.config/dvc
        System: /etc/dvc
Cache types: hardlink, symlink
Cache directory: zfs on zvol1/home/aaron
Caches: local
Remotes: s3, s3
Workspace directory: zfs on zvol1/home/aaron
Repo: dvc, git
Repo.site_cache_dir: /var/tmp/dvc/repo/f92fa966085d661846d8ea2e53107206

Additional Information (if any):

Here's the output from dvc push -r wasabi --verbose <filename>: output.txt

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugDid we break something?fs: s3Related to the S3 filesystemupstreamIssues which need to be resolved in an upstream dependency

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions