rbd: merge diff error

[aaronxrf]

Greetings,
We now face a unknown to us error during backup diff merging process.
On all scheduled backup jobs.

Command used to run:
eve4pve-barc backup --vmid=VMID --label='weekly' --path='/my-mount-point' --keep=4

Output of command is a follows:
ACTION: Backup
VM VMID - ======== Start backup VMID VMID ========
VM VMID - -------- Snapshots Disks VMID VMID--------
VM VMID - Renew policy tolerates incremental run for vmpool:vm-VMID-disk-0
VM VMID - Creating snapshot vmpool/vm-VMID-disk-0@barcweekly20200911210026
VM VMID - -------- Export images --------
VM VMID - Export diff '/my-mount-point/barc/VMID/weekly/20200911210026vmpool.vm-VMID-disk-0.diff'
VM VMID - excmd: rbd export-diff --from-snap barcweekly20200905010645 vmpool/vm-VMID-disk-0@barcweekly20200911210026 - | tee >(cat > '/my-mount-point/barc/VMID/weekly/20200911210026vmpool.vm-VMID-disk-0.diff') | tee >({ wc -c; } > /my-mount-point/barc/VMID/weekly/20200911210026vmpool.vm-VMID-disk-0.diff.sha1.size) | sha1sum > /my-mount-point/barc/VMID/weekly/20200911210026vmpool.vm-VMID-disk-0.diff.sha1

Exporting image: 1% complete...
Exporting image: 100% complete...done.
VM VMID - Remove snapshot vmpool/vm-VMID-disk-0@barcweekly20200905010645

Removing snap: 100% complete...done.
VM VMID - Size 234.74 MiB Uncompressed: 234.74 MiB Compressionratio 1:1
VM VMID - -------- Copy config --------
VM VMID - -------- Merge diff file --------
VM VMID - Merge diff backup '20200821210024vmpool.vm-VMID-disk-0.diff' '20200828210024vmpool.vm-VMID-disk-0.diff'
invalid banner 'ëc ŽÐ¼', expected 'rbd diff v1
'
rbd: failed to parse first diff header

Merging image diff: 0% complete...failed.
rbd: merge-diff error
ERROR: rbd merge-diff '/my-mount-point/barc/VMID/weekly/20200821210024vmpool.vm-VMID-disk-0.diff' '/my-mount-point/barc/VMID/weekly/20200828210024vmpool.vm-VMID-disk-0.diff' '/my-mount-point/barc/VMID/weekly/20200828210024vmpool.vm-VMID-disk-0.diff-merged'
ERROR: VM VMID - Merge diff backup '20200821210024vmpool.vm-VMID-disk-0.diff' '20200828210024vmpool.vm-VMID-disk-0.diff' to '20200828210024vmpool.vm-VMID-disk-0.diff-merged'
End backup 2020-09-11 21:01:01

The target share have quota available and storage space also.
Please advice what should be done.
Renaming the target "weekly" folder and re-running the command two times creates new backups without this errors.

[lephisto]

This looks like the backupchain is incomplete.

[iwen65]

Update 2: I had luck with my first VM. I tested 8 meanwhile: 6 from 8 Restores returns with: ERROR: VM XXX - Checksum missmatch. Check chain. (6b4e6df392409e040a255fee6c221c04acef5ace - / 015b187340cd079fb1404e01c2518817720fa05e -)

Update: I just tried the very latest 2.8 and it seems the older backup could imported again. Will keep you updated in some weeks.

Hello aaron, we had the same problem. We initially suspected it was a consequence of an Ceph update. But we are no longer sure. All long-term backups were unusable. Maybe a problem when merging the base and the oldest diff.

We then installed the latest version, deleted all old backups and started over. Some days everything was ok. But now, after several month, again no backup could be restored.

I would currently not rely on the backup.