[BUG]: The current version of s3 integration probably is not working with 3rd-party s3 compatible services (like GCS CloudFlare R2)

[yxzhao6]

Describe the bug

aws s3 --endpoint-url https://storage.googleapis.com cp response.json s3://yx-test-3/response.json
upload failed: ./response.json to s3://yx-test-3/response.json An error occurred (SignatureDoesNotMatch) when calling the PutObject operation: Invalid argument.

Repro MVP (Minimal Viable Procedure)

• Bucket creation on GCS
• Enabling Interoperability access
• Creating access keys
• Using the S3-compatible endpoint
• Verifying access with an S3 client

Expected behavior

Upload succeeded

Screenshots

Versions (please complete the following information):

• Docker version: [e.g. 20.10.7]
• ComfyUI version: [e.g. 1.2.0]
• Host OS: [e.g. Ubuntu 20.04]

Additional context

[yxzhao6]

terricain/aioboto3#358

This might be the relevant issue.

[yxzhao6]

Below, I show that boto3 1.35.40 works fine, boto3==1.40.1 breaks the code:

GCS_ENDPOINT_URL = "https://yx-test-3.storage.googleapis.com"
s3_client = boto3.client(
    "s3",
    # access id and key are set in environment variables
    endpoint_url=GCS_ENDPOINT_URL,
    config=Config(signature_version="s3v4"),
)

# --- Define a local file and its destination in the bucket ---
local_file_path = "req.json"
object_key = "req.json"
downloaded_file_path = "downloaded_req.json"

s3_client.upload_file(local_file_path, BUCKET_NAME, object_key)

(.venv) [22:04:33] yzhao:comfyui git:(main*) $ pip install boto3==1.35.40

Collecting boto3==1.35.40
Using cached boto3-1.35.40-py3-none-any.whl (139 kB)
Collecting botocore<1.36.0,>=1.35.40 (from boto3==1.35.40)
Using cached botocore-1.35.99-py3-none-any.whl (13.3 MB)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in ./.venv/lib/python3.11/site-packages (from boto3==1.35.40) (1.0.1)
Collecting s3transfer<0.11.0,>=0.10.0 (from boto3==1.35.40)
Using cached s3transfer-0.10.4-py3-none-any.whl (83 kB)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in ./.venv/lib/python3.11/site-packages (from botocore<1.36.0,>=1.35.40->boto3==1.35.40) (2.9.0.post0)
Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in ./.venv/lib/python3.11/site-packages (from botocore<1.36.0,>=1.35.40->boto3==1.35.40) (2.5.0)
Requirement already satisfied: six>=1.5 in ./.venv/lib/python3.11/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.36.0,>=1.35.40->boto3==1.35.40) (1.17.0)
Installing collected packages: botocore, s3transfer, boto3
Attempting uninstall: botocore
Found existing installation: botocore 1.40.1
Uninstalling botocore-1.40.1:
Successfully uninstalled botocore-1.40.1
Attempting uninstall: s3transfer
Found existing installation: s3transfer 0.13.1
Uninstalling s3transfer-0.13.1:
Successfully uninstalled s3transfer-0.13.1
Successfully installed boto3-1.35.40 botocore-1.35.99 s3transfer-0.10.4

[notice] A new release of pip is available: 23.1.2 -> 25.2
[notice] To update, run: pip install --upgrade pip
(.venv) [22:04:46] yzhao:comfyui git:(main*) $ python gcs_s3.py
(.venv) [22:04:54] yzhao:comfyui git:(main*) $ pip install boto3 --upgrad
e
Requirement already satisfied: boto3 in ./.venv/lib/python3.11/site-packages (1.35.40)
Collecting boto3
Using cached boto3-1.40.1-py3-none-any.whl (139 kB)
Collecting botocore<1.41.0,>=1.40.1 (from boto3)
Using cached botocore-1.40.1-py3-none-any.whl (13.9 MB)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in ./.venv/lib/python3.11/site-packages (from boto3) (1.0.1)
Collecting s3transfer<0.14.0,>=0.13.0 (from boto3)
Using cached s3transfer-0.13.1-py3-none-any.whl (85 kB)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in ./.venv/lib/python3.11/site-packages (from botocore<1.41.0,>=1.40.1->boto3) (2.9.0.post0)
Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in ./.venv/lib/python3.11/site-packages (from botocore<1.41.0,>=1.40.1->boto3) (2.5.0)
Requirement already satisfied: six>=1.5 in ./.venv/lib/python3.11/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.41.0,>=1.40.1->boto3) (1.17.0)
Installing collected packages: botocore, s3transfer, boto3
Attempting uninstall: botocore
Found existing installation: botocore 1.35.99
Uninstalling botocore-1.35.99:
Successfully uninstalled botocore-1.35.99
Attempting uninstall: s3transfer
Found existing installation: s3transfer 0.10.4
Uninstalling s3transfer-0.10.4:
Successfully uninstalled s3transfer-0.10.4
Attempting uninstall: boto3
Found existing installation: boto3 1.35.40
Uninstalling boto3-1.35.40:
Successfully uninstalled boto3-1.35.40
Successfully installed boto3-1.40.1 botocore-1.40.1 s3transfer-0.13.1

[notice] A new release of pip is available: 23.1.2 -> 25.2
[notice] To update, run: pip install --upgrade pip
(.venv) [22:05:05] yzhao:comfyui git:(main*) $ python gcs_s3.py
Traceback (most recent call last):
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/.venv/lib/python3.11/site-packages/boto3/s3/transfer.py", line 372, in upload_file
future.result()
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/.venv/lib/python3.11/site-packages/s3transfer/futures.py", line 111, in result
return self._coordinator.result()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/.venv/lib/python3.11/site-packages/s3transfer/futures.py", line 287, in result
raise self._exception
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/.venv/lib/python3.11/site-packages/s3transfer/tasks.py", line 142, in call
return self._execute_main(kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/.venv/lib/python3.11/site-packages/s3transfer/tasks.py", line 165, in _execute_main
return_value = self._main(**kwargs)
^^^^^^^^^^^^^^^^^^^^
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/.venv/lib/python3.11/site-packages/s3transfer/upload.py", line 796, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/.venv/lib/python3.11/site-packages/botocore/client.py", line 601, in _api_call
return self._make_api_call(operation_name, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/.venv/lib/python3.11/site-packages/botocore/context.py", line 123, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/.venv/lib/python3.11/site-packages/botocore/client.py", line 1074, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (SignatureDoesNotMatch) when calling the PutObject operation: Invalid argument.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/gcs_s3.py", line 33, in
s3_client.upload_file(local_file_path, BUCKET_NAME, object_key)
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/.venv/lib/python3.11/site-packages/botocore/context.py", line 123, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/.venv/lib/python3.11/site-packages/boto3/s3/inject.py", line 175, in upload_file
return transfer.upload_file(
^^^^^^^^^^^^^^^^^^^^^
File "/Users/yzhao/Workspace/NascentCore/inty-backend/experimental/comfyui/.venv/lib/python3.11/site-packages/boto3/s3/transfer.py", line 378, in upload_file
raise S3UploadFailedError(
boto3.exceptions.S3UploadFailedError: Failed to upload req.json to yx-test-3/req.json: An error occurred (SignatureDoesNotMatch) when calling the PutObject operation: Invalid argument.

[yxzhao6]

The fix could be https://g.co/gemini/share/551d4649553d

[yxzhao6]

Adding these 2 env vars actually fixed the issue.

I'd suggest the team include these 2 env vars or note them in the doc.