Skip to content

Misleading error on wrong credentials #6281

@Cenness

Description

@Cenness

What happened:
If s3 key ends with - Format fails with

2025/07/24 09:30:31.086205 juicefs[7] <WARNING>:
 Test storage minio://http://juice-minio.cicd-tools.svc.cluster.local:9000/juicefs-dev-kngf/juicefs-dev-kngf/juicefs-dev-vol/ failed:
 Failed to create bucket minio://http://juice-minio.cicd-tools.svc.cluster.local:9000/juicefs-dev-kngf/juicefs-dev-kngf/juicefs-dev-vol/:
 operation error S3: CreateBucket, https response error StatusCode: 403, RequestID: 1855260497119D07, HostID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8,
 api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.,
 previous error: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 1855260496DBFBB2, HostID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method., tries: #1 [[email protected]:374]

What you expected to happen:
error about credentials specifically

How to reproduce it (as minimally and precisely as possible):
install latest minio:

apiVersion: v1
kind: Namespace
metadata:
  name: cicd-tools
---
apiVersion: v1
kind: Service
metadata:
  name: juice-minio
  namespace: cicd-tools
  labels:
    app: juice-minio
spec:
  ports:
    - port: 9000
      protocol: TCP
  selector:
    app: juice-minio
  type: ClusterIP
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: juice-minio
  namespace: cicd-tools
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 8Gi
---
apiVersion: v1
kind: Secret
metadata:
  name: juice-minio
  namespace: cicd-tools
type: Opaque
stringData:
  access-key: juicefsjuicefs
  secret-key: juicefsjuicefs
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: juice-minio  
  namespace: cicd-tools
spec:
  selector:
    matchLabels:
      app: juice-minio
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: juice-minio
    spec:
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: juice-minio
      containers:
      - name: minio
        volumeMounts:
        - name: data 
          mountPath: "/data"
        image: minio/minio:RELEASE.2025-07-23T15-54-02Z
        args:
        - server
        - /data
        env:
        - name: MINIO_ACCESS_KEY
          valueFrom:
            secretKeyRef:
              name: juice-minio
              key: access-key
        - name: MINIO_SECRET_KEY
          valueFrom:
            secretKeyRef:
              name: juice-minio
              key: secret-key
        ports:
        - containerPort: 9000
        readinessProbe:
          httpGet:
            path: /minio/health/ready
            port: 9000
          periodSeconds: 20
        livenessProbe:
          httpGet:
            path: /minio/health/live
            port: 9000
          initialDelaySeconds: 30
          periodSeconds: 20

install Juice CSI
create test storageclass, pvc and pod:

apiVersion: v1
kind: Secret
metadata:
  name: juicefs-secret
  namespace: kube-system
type: Opaque
stringData:
  name: juicefs-dev-vol
  access-key: juicefsjuicefs
  secret-key: juicefsjuicefs- # typo here
  metaurl: redis://valk.cicd-tools.svc.cluster.local:6379/1
  storage: minio
  bucket: http://juice-minio.cicd-tools.svc.cluster.local:9000/juicefs-dev-kngf
  format-options: trash-days=0
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: juicefs-sc
provisioner: csi.juicefs.com
parameters:
  csi.storage.k8s.io/provisioner-secret-name: juicefs-secret
  csi.storage.k8s.io/provisioner-secret-namespace: kube-system
  csi.storage.k8s.io/node-publish-secret-name: juicefs-secret
  csi.storage.k8s.io/node-publish-secret-namespace: kube-system
  csi.storage.k8s.io/controller-expand-secret-name: juicefs-secret
  csi.storage.k8s.io/controller-expand-secret-namespace: kube-system
  pathPattern: "${.pvc.namespace}-${.pvc.name}"
allowVolumeExpansion: true
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: juicefs-pvc
  namespace: default
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: juicefs-sc
---
apiVersion: v1
kind: Pod
metadata:
  name: juicefs-app
  namespace: default
spec:
  containers:
  - args:
    - -c
    - while true; do echo $(date -u) >> /data/out.txt; sleep 5; done
    command:
    - /bin/sh
    image: busybox
    name: app
    volumeMounts:
    - mountPath: /data
      name: juicefs-pv
  volumes:
  - name: juicefs-pv
    persistentVolumeClaim:
      claimName: juicefs-pvc

Anything else we need to know?

Environment:

  • JuiceFS version (use juicefs --version) or Hadoop Java SDK version: 1.3.0+2025-07-03.30190ca1
  • Cloud provider or hardware configuration running JuiceFS: N/A
  • OS (e.g cat /etc/os-release): Debian GNU/Linux 11 (bullseye)
  • Kernel (e.g. uname -a): Linux juicefs-csi-controller-1 5.15.0-83-generic #92-Ubuntu SMP Mon Aug 14 09:30:42 UTC 2023 x86_64 GNU/Linux
  • Object storage (cloud provider and region, or self maintained): minio compatible
  • Metadata engine info (version, cloud provider managed or self maintained): valkey 8.1.3
  • Network connectivity (JuiceFS to metadata engine, JuiceFS to object storage): Ok

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-more-infoThis issue requires more information to address

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions