remote repository exceptions give classic (not modern) result codes

[kirscheGIT]

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

ISSUE

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

1.4.0 on server and client

Operating system (distribution) and version.

Client:
Distributor ID: Debian
Description: Debian GNU/Linux 12 (bookworm)
Release: 12
Codename: bookworm

Server: borgbase

Full borg commandline that lead to the problem (leave away excludes and passwords)

borg create --info --stats --compression=auto,zstd,9 --files-cache=ctime,size,inode --show-rc ssh://testrepo_borgbase/./repo_modern_rc::daily-20250111 /path/to/myfilestobackup

Describe the problem you're observing.

I'm writing some wrapper scripts for borg. I want to rely on the return codes of borg with the modern BORG_EXIT_CODES, to trigger the init of repo, if it doesn't exist.
I've exported the BORG_EXIT_CODES=modern variable.
But when I try to execute above command line, I'll get rc 2 instead of rc 15

is not a valid repository. Check repo config.
terminating with error status, rc 2

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

If I do the same with a local existing path without repository, I'll get rc 15

borg create --info --stats --compression=auto,zstd,9 --files-cache=ctime,size,inode --show-rc /home/borg/test_repo_rc::daily-20250111  /path/to/myfilestobackup
/home/borg/test_repo_rc is not a valid repository. Check repo config.
terminating with error status, rc 15

I've did the same test with a remote server at Hetzner.

borg create --info --stats --compression=auto,zstd,9 --files-cache=ctime,size,inode --show-rc --remote-path=borg ssh://borg_hetzner_sub2/home/test_repo_rc::daily-20250111 /path/to/myfilestobackup

I'll get rc 2, too.

When I try to create the repo on a directory which doesn't exist I'll get rc 13 on the remote Hetzner box. Which I would expect.

My assumption is, that the handling for invalid repository error code some how differs between local and remote repos

[infectormp]

Please show an example of exporting variables.

[kirscheGIT]

@infectormp Thanks for your answer.
Here is the export command and the test with echo

borg@testmachine:~$ export BORG_EXIT_CODES=modern
borg@testmachine:~$ echo $BORG_EXIT_CODES
modern

And here is the complete manual test. First local and then with repositories located at hetzner, borgbase and rsync.net

borg@testmachine:~$ echo $BORG_EXIT_CODES

borg@testmachine:~$ export BORG_EXIT_CODES=modern
borg@testmachine:~$ echo $BORG_EXIT_CODES
modern

borg@testmachine:~$ echo "Test with local repository - empty uninitialized folder exists"
Test with local repository - empty uninitialized folder exists
borg@testmachine:~$ borg create --info --stats --show-rc /home/borg/test_repo_rc::daily-20250111 /tmp/rc_modern/
/home/borg/test_repo_rc is not a valid repository. Check repo config.
terminating with error status, rc 15

borg@testmachine:~$ echo "Test with local repository - empty uninitialized folder doesn't exist"
Test with local repository - empty uninitialized folder doesn't exist
borg@testmachine:~$ borg create --info --stats --show-rc /home/borg/test_repo_rc_not_created::daily-20250111 /tmp/rc_modern/
Repository /home/borg/test_repo_rc_not_created does not exist.
terminating with error status, rc 13

borg@testmachine:~$ echo "For local repository this looks good"
For local repository this looks good

borg@testmachine:~$ echo "Test with hetzner repo - empty uninitialized folder exists"
Test with hetzner repo - empty uninitialized folder exists
borg@testmachine:~$ borg create --info --stats --show-rc ssh://hetzner_repo/home/test_repo_rc::daily-20250111 /tmp/rc_modern/
/home/test_repo_rc is not a valid repository. Check repo config.
terminating with error status, rc 2

borg@testmachine:~$ echo "Test with hetzner repository - empty uninitialized folder doesn't exist"
Test with hetzner repository - empty uninitialized folder doesn't exist
borg@testmachine:~$ borg create --info --stats --show-rc ssh://hetzner_repo/home/test_repo_rc_not_created::daily-20250111 /tmp/rc_modern/
Repository ssh://hetzner_repo/home/test_repo_rc_not_created does not exist.
terminating with error status, rc 13

borg@testmachine:~$ echo "For hetzner repo we see the problem"
For hetzner repo we see the problem

borg@testmachine:~$ echo "Test with borgbase repo - empty uninitialized folder exists"
Test with borgbase repo - empty uninitialized folder exists
borg@testmachine:~$ borg create --info --stats --show-rc ssh://borgbase_repo/./repo::daily-20250111 /tmp/rc_modern/
/srv/repos/username/repo is not a valid repository. Check repo config.
terminating with error status, rc 2

borg@testmachine:~$ echo "Test with borgbase repository - empty uninitialized folder doesn't exist"
Test with borgbase repository - empty uninitialized folder doesn't exist
borg@testmachine:~$ borg create --info --stats --show-rc ssh://borgbase_repo/./repo_rc_not_created::daily-20250111 /tmp/rc_modern/
Repository path not allowed: /srv/repos/username/repo_rc_not_created
terminating with error status, rc 83

borg@testmachine:~$ echo "For borgbase repo we see the problem"
For borgbase repo we see the problem

borg@testmachine:~$ echo "Test with rsync.net repo - empty uninitialized folder exists"
Test with rsync.net repo - empty uninitialized folder exists
borg@testmachine:~$ borg create --info --stats --show-rc ssh://rsync_net_repo/./test_repo_rc::daily-20250111 /tmp/rc_modern/
/data1/home/username/test_repo_rc is not a valid repository. Check repo config.
terminating with error status, rc 2

borg@testmachine:~$ borg create --info --stats --show-rc ssh://rsync_net_repo/./test_repo_rc_not_created::daily-20250111 /tmp/rc_modern/
Repository ssh://rsync_net_repo/./test_repo_rc_not_created does not exist.
terminating with error status, rc 13

borg@testmachine:~$ echo $BORG_EXIT_CODES
modern

As you can see, as soon as I use an empty uninitialized folder on a remote location I'll get rc 2. In contrast on the local path I'll get rc 15, which is what I would expect.

[infectormp]

It's happens because variables were set for local environment. To get modern rc codes from remotely executed borg you need to set variables on remote location.

[kirscheGIT]

Then why do I get rc 13 for the remote repository?
When I disable the modern rc I get instead rc 2 for local and remote.
And why is the error text for the remote and local repository the same?

...is not a valid repository. Check repo config.

[kirscheGIT]

@infectormp To double check this, I've created a virtual machine as borg server. First with

command="borg serve", restrict ...

and then with

command="export BORG_EXIT_MODES=modern; borg serve", restrict ...

In both cases the behavior is exactly the same:
rc 2 for an empty directory
rc 13 for a none existing directory

borg@testmachine:~/.ssh$ borg create --info --stats --show-rc ssh://test_borg_server/./test_repo_rc::daily-20250111 /tmp/rc_modern/
/home/borg/test_repo_rc is not a valid repository. Check repo config.
terminating with error status, rc 2
borg@testmachine:~/.ssh$ borg create --info --stats --show-rc ssh://test_borg_server/./repo_rc_not_created::daily-20250111 /tmp/rc_modern/
Repository ssh://test_borg_server/./repo_rc_not_created does not exist.
terminating with error status, rc 13

[kirscheGIT]

@infectormp As I'm not familiar with Python, I might be wrong, but I think I've found the code which causes this behavior.

What I understand is that certain remote errors have an exemption from the error handler on the server side. Those errors are passed to the client and the modern error code is than reconstructed on the client side. There the modern error code is than used.
And here is the InvalidRepository missing.

borg/src/borg/remote.py

Line 253 in 8b7cce3

except BaseException as e:

except BaseException as e:
                        if dictFormat:
                            ex_short = traceback.format_exception_only(e.__class__, e)
                            ex_full = traceback.format_exception(*sys.exc_info())
                            ex_trace = True
                            if isinstance(e, Error):
                                ex_short = [e.get_message()]
                                ex_trace = e.traceback
                            if isinstance(e, (Repository.DoesNotExist, Repository.AlreadyExists, PathNotAllowed)):
                                # These exceptions are reconstructed on the client end in RemoteRepository.call_many(),
                                # and will be handled just like locally raised exceptions. Suppress the remote traceback
                                # for these, except ErrorWithTraceback, which should always display a traceback.
                                pass
                            else:
                                logging.debug('\n'.join(ex_full))

I think the if call has to look like this, to bring up the correct error code on the client:

if isinstance(e, (Repository.InvalidRepository, Repository.DoesNotExist, Repository.AlreadyExists, PathNotAllowed)):

What do you think?

[infectormp]

We'd better ask @ThomasWaldmann

[ThomasWaldmann]

Guess it is a problem with the RPC layer, how it transports exceptions from remote to local.

[ThomasWaldmann]

@kirscheGIT I just noticed a typo there: #8631 (comment)

Instead of BORG_EXIT_CODES there is ...MODES.

Just on github or was the typo also on the server you tried that?

Update: maybe not important, I suspect that does not influence the behavior.