velero backup logs or describe is not working

[wangshuai0709]

velero backup logs tektonbackup

An error occurred: Get "http://minio.velero.svc.cluster.local:9000/velero/backups/tektonbackup/tektonbackup-logs.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio%2F20250630%2Fminio%2Fs3%2Faws4_request&X-Amz-Date=20250630T084320Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=af8a288cc0865bab2be634eaba6fea545b3e04c44d9bce81e9e43d4432ad4edb": dial tcp: lookup minio.velero.svc.cluster.local on 203.93.119.33:53: read udp 10.113.208.15:55153->203.93.119.33:53: i/o timeout

The minio.velero.svc.cluster.local can be lookup in other pods:

Why the error repoted by velero backup log?

[kaovilai]

Duplicate of #6167

[wangshuai0709]

I didn't get any useful information from 6167.

[wangshuai0709]

I don't know what to do.

[kaovilai]

#6167 proposes solution to your issue.

graph TB
    subgraph "Current Architecture (Problem)"
        Client1[Velero CLI Client]
        ObjectStore1[Object Storage<br/>S3/GCS/Azure]
        
        Client1 -->|Direct Access Required| ObjectStore1
        
        Client1 -.->|"❌ May not have access<br/>(firewall/network isolation)"| ObjectStore1
        Client1 -.->|"❌ Requires credentials<br/>--cacert or --insecure"| ObjectStore1
        Client1 -.->|"❌ Assumes object store<br/>(won't work with NFS/Unified Repo)"| ObjectStore1
    end
    
    subgraph "Proposed Solution"
        Client2[Velero CLI Client]
        VeleroServer[Velero Server<br/>Download Endpoint]
        K8sAPI[Kubernetes API Server]
        ObjectStore2[Object Storage]
        NFS[NFS Storage]
        UnifiedRepo[Unified Repo]
        
        Client2 -->|"kubectl-style<br/>communication"| K8sAPI
        K8sAPI -->|API Aggregation| VeleroServer
        VeleroServer -->|Storage Agnostic<br/>Data Access Layer| ObjectStore2
        VeleroServer -->|Storage Agnostic<br/>Data Access Layer| NFS
        VeleroServer -->|Storage Agnostic<br/>Data Access Layer| UnifiedRepo
    end
    
    subgraph "Commands Affected"
        Commands["velero backup download/logs/describe"]
    end
    
    Commands -.->|Currently require| Client1
    Commands -.->|Will use| Client2
    
    style Client1 fill:#ff6b6b
    style ObjectStore1 fill:#ffd93d
    style Client2 fill:#51cf66
    style VeleroServer fill:#4dabf7
    style K8sAPI fill:#868e96
    style Commands fill:#f8f9fa,stroke:#868e96,stroke-width:2px

Loading

The diagram shows:

Current Problem (Top Section):

• The Velero CLI client directly accesses object storage for commands like velero backup download, velero describe, and velero backup logs
• This creates several issues:
  • The client may not have network access to the object store (common in on-premise environments)
  • Requires separate credential management (--cacert or --insecure flags)
  • Assumes storage is always object-based (won't work with future NFS or Unified Repo implementations)

Proposed Solution (Bottom Section):

• Introduces a Velero Server Download Endpoint that acts as a proxy
• The client communicates through the Kubernetes API (using API Aggregation Layer)
• The server has a storage-agnostic data access layer that can work with:
  • Object Storage (S3/GCS/Azure)
  • NFS Storage
  • Future Unified Repository implementations
• Benefits:
  • Client only needs access to Kubernetes API (which it already has)
  • No direct storage credentials needed on client side
  • Storage type is transparent to the client
  • Works in network-isolated environments

Kubectl Exec Workaround

To avoid the current issue while the download server is being developed, you can use:

kubectl exec deploy/velero -n velero -- /velero backup logs <backup-name>

This workaround (#9001):

• Executes the velero command inside the Velero pod itself
• The pod has direct access to object storage with proper credentials
• Avoids the network access issues from the client side

For other commands:

# For describe commands
kubectl exec deploy/velero -n velero -- /velero describe backup <backup-name>

This approach leverages the fact that the Velero server pod already has the necessary network access and credentials to reach the object storage, effectively using it as a proxy until the official download server solution is implemented.

[kaovilai]

TL;DR

Problem: Your MinIO object storage is only accessible from within the Kubernetes cluster (using an internal ClusterIP or private network address). When you run velero CLI commands from outside the cluster, it can't reach MinIO to download backups, logs, or descriptions.

Quick Fix: Run the velero command inside the cluster where it can reach MinIO:

Why this works: The Velero server pod inside the cluster can access MinIO's internal address, while your local machine cannot. Using kubectl exec runs the command inside the pod, bypassing the network isolation issue.

Long-term solution: Velero team is working on a download server (issue #6167) that will proxy these requests through the Kubernetes API, eliminating the need for direct object storage access from the CLI.

[wangshuai0709]

Thank you very mush.I see.