Large-scale repack after switching from restic?

[lephyrus]

I have a repo of ~600 GiB and I've decided to switch from restic to using the rustic client. (Side note: I had to do a restore for the first time and the rustic feature that made me switch is resumable restores.)

It seems to me that rustic wants to repack almost half the data.

# rustic prune -n

to repack:      17797 packs,     255710 blobs,  285.9 GiB
-- snip --
unused size after prune:    5.2 GiB (0.84% of remaining size)

I'm not clear on why, as the unused size is below 1% even without repacking.

# rustic prune -n --max-repack 0

to repack:          0 packs,          0 blobs,        0 B
-- snip --
unused size after prune:    5.2 GiB (0.84% of remaining size)

Am I wrong in my understanding that repacking should kick in only once the unused size is above 5% (per default)?

I've also been slightly confused about --max-repack, which doesn't have the effect I expect when used with a percentage value:

# rustic prune -n --max-repack 10%

to repack:      17805 packs,     255815 blobs,  286.1 GiB

Is this not the right way to pass a percentage value?

[aawsome]

There is actually a somewhat hidden feature in rustic that the packsize can be configured for the repository and depends on a fixed size plus a growfactor which increases the packsize depending on the actual repository size.
You can tweak this for the repository using the rustic config command.

Another effect is that prune can be configured to repack packs which are too small or too large. The thresholds can also be configured using rustic config. Defaults are to repack too small packs but to keep too large packs.

TL;DR: I think in your case rustic prune computes a larger targeted pack size as restic used to use and the prune command wants to repack the too-small packs.

About --max-repack: This should actually work. If you specify a percentage, IIRC the the percent of total repository size which you allow to repack.

[lephyrus]

Thanks for the quick reply.

TL;DR: I think in your case rustic prune computes a larger targeted pack size as restic used to use and the prune command wants to repack the too-small packs.

Yep, that was it. After a rustic config --set-min-packsize-tolerate-percent 25 (default: 30), rustic only wants to repack 31 packs (down from 17805).

As an end-user who's not aware of the ins and outs of the repository concepts, that was unexpected and slightly worrying behaviour. Might be worth documenting somewhere—well, I guess there's this discussion now, at least.

About --max-repack: This should actually work. If you specify a percentage, IIRC the the percent of total repository size which you allow to repack.

Definitely not working for me, but maybe it's caused by a specific set of circumstances.

[aawsome]

About --max-repack: This should actually work. If you specify a percentage, IIRC the the percent of total repository size which you allow to repack.

Definitely not working for me, but maybe it's caused by a specific set of circumstances.

Ah, yes - you found a bug, see #648.

[lephyrus]

Oh, one more question: Is it possible to put the min-packsize-tolerate-percent into the config file? I don't see it in https://github.com/rustic-rs/rustic/blob/main/examples/full.toml.

[aawsome]

The options that you can set via the rustic config command are per-repository options as they describe how packs are written to the repository (i.e., currently compression and packsize-options). They are saved in the repository config file and are automatically used by rustic once they are set for the repository. So, no need to specify them in the clients config file.

The options that can be given on the command line or the config file on the other hand are per-client options like which repositories to talk to or which backup sources are used with which options. For those the config file is the recommended place to put the options.

[lephyrus]

Ah, that makes sense. I was wondering where rustic had persisted that setting. Thanks for the explanation.

[rmsc]

Related question, is there a way to print the current repository configuration?

I've tried rustic show-config, but that just seems to print the client configuration.

[aawsome]

You can run rustic cat config. This shows the current config as it is saved, i.e. in JSON format.