Skip to content

Commit ff9e49c

Browse files
committed
Fix: Enable throttling parameters in BackupRepository repositoryConfig
The `uploadBytes`, `downloadBytes`, and other throttling parameters specified in BackupRepository's `repositoryConfig` were being silently filtered out and not applied to Kopia repositories. This occurred because these parameters were missing from the whitelist in `getStorageVariables()`. **Code:** - Added throttling options to the valid parameters whitelist in `pkg/repository/provider/unified_repo.go`: - `uploadBytes` - upload bandwidth throttling (bytes/sec) - `downloadBytes` - download bandwidth throttling (bytes/sec) - `readOPS` - read operations throttling (ops/sec) - `writeOPS` - write operations throttling (ops/sec) - `listOPS` - list operations throttling (ops/sec) **Documentation:** - Updated `site/content/docs/main/backup-repository-configuration.md` with throttling parameters descriptions and examples **Tests:** - Added test case in `pkg/repository/provider/unified_repo_test.go` to verify throttling parameters are properly passed through ```yaml apiVersion: velero.io/v1 kind: BackupRepository spec: repositoryConfig: uploadBytes: "838860800" # ~800 MB/s downloadBytes: "838860800" # ~800 MB/s ``` These parameters are now properly applied to Kopia during file system backups performed by node-agent. Signed-off-by: Andrei Kvapil <[email protected]>
1 parent 99f12b8 commit ff9e49c

File tree

4 files changed

+50
-1
lines changed

4 files changed

+50
-1
lines changed

changelogs/unreleased/9351-kvaps

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Fix: Enable throttling parameters in BackupRepository repositoryConfig

pkg/repository/provider/unified_repo.go

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -586,6 +586,11 @@ func getStorageVariables(backupLocation *velerov1api.BackupStorageLocation, repo
586586
validParams := []string{
587587
udmrepo.StoreOptionCacheLimit,
588588
udmrepo.StoreOptionKeyFullMaintenanceInterval,
589+
udmrepo.ThrottleOptionReadOps,
590+
udmrepo.ThrottleOptionWriteOps,
591+
udmrepo.ThrottleOptionListOps,
592+
udmrepo.ThrottleOptionUploadBytes,
593+
udmrepo.ThrottleOptionDownloadBytes,
589594
}
590595
for _, param := range validParams {
591596
if v, found := backupRepoConfig[param]; found {

pkg/repository/provider/unified_repo_test.go

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -460,6 +460,37 @@ func TestGetStorageVariables(t *testing.T) {
460460
"cacheLimitMB": "1000",
461461
},
462462
},
463+
{
464+
name: "fs with throttle config",
465+
backupLocation: velerov1api.BackupStorageLocation{
466+
Spec: velerov1api.BackupStorageLocationSpec{
467+
Provider: "velero.io/fs",
468+
Config: map[string]string{
469+
"fspath": "fake-path",
470+
"prefix": "fake-prefix",
471+
},
472+
},
473+
},
474+
repoBackend: "fake-repo-type",
475+
repoConfig: map[string]string{
476+
udmrepo.ThrottleOptionUploadBytes: "838860800",
477+
udmrepo.ThrottleOptionDownloadBytes: "838860800",
478+
udmrepo.ThrottleOptionReadOps: "100",
479+
udmrepo.ThrottleOptionWriteOps: "100",
480+
udmrepo.ThrottleOptionListOps: "50",
481+
},
482+
expected: map[string]string{
483+
"fspath": "fake-path",
484+
"bucket": "",
485+
"prefix": "fake-prefix/fake-repo-type/",
486+
"region": "",
487+
"uploadBytes": "838860800",
488+
"downloadBytes": "838860800",
489+
"readOPS": "100",
490+
"writeOPS": "100",
491+
"listOPS": "50",
492+
},
493+
},
463494
}
464495

465496
for _, tc := range testCases {

site/content/docs/main/backup-repository-configuration.md

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,9 @@ data:
3131
<kopia>: |
3232
{
3333
"cacheLimitMB": 2048,
34-
"fullMaintenanceInterval": "fastGC"
34+
"fullMaintenanceInterval": "fastGC",
35+
"uploadBytes": "838860800",
36+
"downloadBytes": "838860800"
3537
}
3638
<other-repository-type>: |
3739
{
@@ -59,5 +61,15 @@ Per kopia [Maintenance Safety](https://kopia.io/docs/advanced/maintenance/#maint
5961

6062
On the other hand, the not-in-use data will be deleted permanently after the full maintenance, so shorter full maintenance intervals may weaken the data safety if they are used incorrectly.
6163

64+
`uploadBytes`: specifies the maximum upload speed in bytes per second for the repository. This can be used to throttle the upload bandwidth. The value should be specified as a string representing bytes per second (e.g., "838860800" for ~800 MB/s). This parameter applies during repository operations that write data.
65+
66+
`downloadBytes`: specifies the maximum download speed in bytes per second for the repository. This can be used to throttle the download bandwidth. The value should be specified as a string representing bytes per second (e.g., "838860800" for ~800 MB/s). This parameter applies during repository operations that read data.
67+
68+
`readOPS`: specifies the maximum number of read operations per second. This can be used to throttle the rate of read operations to the storage backend.
69+
70+
`writeOPS`: specifies the maximum number of write operations per second. This can be used to throttle the rate of write operations to the storage backend.
71+
72+
`listOPS`: specifies the maximum number of list operations per second. This can be used to throttle the rate of list operations to the storage backend.
73+
6274
[1]: file-system-backup.md
6375
[2]: csi-snapshot-data-movement.md

0 commit comments

Comments
 (0)