git: port remote management to gix

[emilazy]

(Plus one more change to remove the other remaining git_util::get_git_repo caller.)

I’m not wholly confident in the first commit here, but the rest should hopefully be good. It feels like there might be awkward race conditions in here but I think Git is just like that.

@Byron 👋 I thought you might be interested in some API UX feedback I came up with while working on this. If you’d like me to open issues/discussions on the gix side for any of these please let me know.

• It’s awkward that gix::Remote::save_to handles the fussy details of removing the other sections for that remote from the config, but leaves you to reimplement it yourself if you need it in any other context. It would be nice if the removal logic was exposed separately, or perhaps if there was a specific way to tell gix that you’re renaming a remote and the old config should go away.

• You can change the name of a gix::Remote, and you can set its push URL, but you can’t change its fetch URL. In set_remote_url I have to manually copy over all the data from the existing remote to achieve this. Not sure if this is a fundamental constraint of the model or just a missing API.

• It’s a little surprising that there’s a rich representation of parsed refspecs but the remote API expects you to pass in a BStr and deal with potentially‐impossible parse errors.

• The borrowing model of configuration and remotes interact awkwardly. In a previous version of this PR where I tried to persist the remote configuration changes to the gix::Repository, I had to clone the entire thing so that I could hold a gix::Remote and a gix::config::SnapshotMut at the same time (otherwise the immutable and mutable borrows conflicted). That felt awkward for a task as simple as adding a remote to the configuration. (In general when using the gix API I kind of wish there was less borrowing and more refcounting/cloning, though I realize that there’s a subjective element to the trade‐off here and that you can wrap a borrow‐happy API in a clone‐happy one but not vice versa.)

• Relatedly, there’s this comment in the code:

  // TODO: make it possible to load snapshots with reloading via .config() and write mutated snapshots back to disk which should be the way
  //       to affect all instances of a repo, probably via `config_mut()` and `config_mut_at()`.

  That would be quite nice for us!

In general there’s more convoluted logic here in the move from git2 to gix, but thankfully most of that is just dealing with lower‐level APIs that don’t bundle configuration changes, ref renames, etc. into one porcelain call. Other than the things listed above, the process went quite smoothly; thanks again for all your work!

Checklist

If applicable:

• I have updated CHANGELOG.md
• I have updated the documentation (README.md, docs/, demos/)
• I have updated the config schema (cli/src/config-schema.json)
• I have added tests to cover my changes

[bsdinis]

Thank you so much for this. Amazing effort!
I'm still working through the patch, will pick it back up tomorrow

[Byron]

Thanks for reeling me in, and sorry for the late response!

• It’s awkward that gix::Remote::save_to handles the fussy details of removing the other sections for that remote from the config, but leaves you to reimplement it yourself if you need it in any other context. It would be nice if the removal logic was exposed separately, or perhaps if there was a specific way to tell gix that you’re renaming a remote and the old config should go away

Indeed, renaming isn't supported yet, and having some of the existing logic available would help implementing it.
Alternatively it would be nice if one could just rename a remote while handling the configuration automatically.

• You can change the name of a gix::Remote, and you can set its push URL, but you can’t change its fetch URL. In set_remote_url I have to manually copy over all the data from the existing remote to achieve this. Not sure if this is a fundamental constraint of the model or just a missing API.

I don't think this is more than an incomplete API.

• It’s a little surprising that there’s a rich representation of parsed refspecs but the remote API expects you to pass in a BStr and deal with potentially‐impossible parse errors.

I think this is me avoiding hassle. What it really wants to do is to allow passing various inputs, including strings, and parse it when needed. Right now there probably is only one API that does this (find_reference()), and that alone gave me quite some trouble already. It's probably still not working intuitively in all common cases.
With that said, other crates are able to pull it off, so I am sure there is a way.
In any case, that's the reason for resorting to AsRef<BStr> (and seeing this tells me that this also is a workaround as typically one would do Into<BStr> if one wants it to work for &str. Probably some tests found AsRef more convenient for their input).

• The borrowing model of configuration and remotes interact awkwardly. In a previous version of this PR where I tried to persist the remote configuration changes to the gix::Repository, I had to clone the entire thing so that I could hold a gix::Remote and a gix::config::SnapshotMut at the same time (otherwise the immutable and mutable borrows conflicted). That felt awkward for a task as simple as adding a remote to the configuration. (In general when using the gix API I kind of wish there was less borrowing and more refcounting/cloning, though I realize that there’s a subjective element to the trade‐off here and that you can wrap a borrow‐happy API in a clone‐happy one but not vice versa.)

The clone-happy APIs in jj are really nice and makes the code so much more 'to the point', and simple-looking. I am loving it.
With that said, gix can still change and maybe it should, to fully embed an Rc/Arc gix::Repository instead of a reference, to fix the issue we see today with &mut and & interactions.
However, I'd hope there can be other ways of dealing with this, and usually I fix these issues once I encounter them myself. After all, the main 'components' of a repository are shared and ref-counted already, so certain types could embed them directly if that avoids some &mut hassle.

In general there’s more convoluted logic here in the move from git2 to gix, but thankfully most of that is just dealing with lower‐level APIs that don’t bundle configuration changes, ref renames, etc. into one porcelain call.

And I am really sorry for this being so half-done right now.
Strangely, these days I find myself working on application code of GitButler and have to resort myself to plumbing code to write Git configuration, for example, or to downright having to use git2 if gitoxide doesn't have the feature yet. One would think that I'd have the time to 'just' add the missing feature, but not really.

Probably for you it's very much the same, but if not, or if there is some room to breathe once the first version of a feature is delivered in jj, maybe there is a chance you could add the API you'd want to solve the problem nicely with gix.
Client code is a great driver for this, it just takes an extra step to get there.

I will try to make these steps for GitButler eventually (sprinkling even more TODO's there 😁), and encourage you to do the same.

[emilazy]

Thanks for the detailed reply, @Byron!

Indeed, renaming isn't supported yet, and having some of the existing logic available would help implementing it. Alternatively it would be nice if one could just rename a remote while handling the configuration automatically.

Higher‐level porcelain APIs would definitely be welcome! But I understand that Git has many layers and there’s a lot of subtlety with higher‐level operations, so just having the lower‐level ones factored in a more reusable way would be great here.

I don't think this is more than an incomplete API.

That’s good to know; I’ll leave a comment to that effect in the code so that it can hopefully be cleaned up later.

I think this is me avoiding hassle. What it really wants to do is to allow passing various inputs, including strings, and parse it when needed. Right now there probably is only one API that does this (find_reference()), and that alone gave me quite some trouble already. It's probably still not working intuitively in all common cases. With that said, other crates are able to pull it off, so I am sure there is a way. In any case, that's the reason for resorting to AsRef<BStr> (and seeing this tells me that this also is a workaround as typically one would do Into<BStr> if one wants it to work for &str. Probably some tests found AsRef more convenient for their input).

Right. I guess you’d want a trait for things that can be converted to refspecs, or maybe you can use TryInto and we could rely on Infallible for the identity case or such…

That said, I think it’d be fine if the remote API just only took pre‐parsed refspecs and you had to explicitly .try_into() if you wanted to parse a string representation of them.

The clone-happy APIs in jj are really nice and makes the code so much more 'to the point', and simple-looking. I am loving it. With that said, gix can still change and maybe it should, to fully embed an Rc/Arc gix::Repository instead of a reference, to fix the issue we see today with &mut and & interactions. However, I'd hope there can be other ways of dealing with this, and usually I fix these issues once I encounter them myself. After all, the main 'components' of a repository are shared and ref-counted already, so certain types could embed them directly if that avoids some &mut hassle.

Right. In this case we get around it by not actually mutating the in‐memory repository configuration (since it isn’t reflected in the ThreadSafeRepository anyway), but otherwise it’s awkward to have a remote wanting a & reference to the repo but the mutable config snapshot needing a &mut. If the remotes could avoid depending on the repository’s lifetime, or if config editing used interior mutability (as I guess it might have to in order to write to a ThreadSafeRepository in the future anyway), then that conflict wouldn’t exist.

In general I think that a bit of refcounting/cloning is usually cheap and can make an API simpler and more general enough that it’s worth it; I would say that lifetimes and error types are probably the trickiest parts of using gitoxide in my experience. But I can also understand wanting to exploit Rust’s capabilities to the fullest and avoid any unnecessary copying.

And I am really sorry for this being so half-done right now. Strangely, these days I find myself working on application code of GitButler and have to resort myself to plumbing code to write Git configuration, for example, or to downright having to use git2 if gitoxide doesn't have the feature yet. One would think that I'd have the time to 'just' add the missing feature, but not really.

Probably for you it's very much the same, but if not, or if there is some room to breathe once the first version of a feature is delivered in jj, maybe there is a chance you could add the API you'd want to solve the problem nicely with gix. Client code is a great driver for this, it just takes an extra step to get there.

I will try to make these steps for GitButler eventually (sprinkling even more TODO's there 😁), and encourage you to do the same.

Yes, the temptation to just dive into the gitoxide code base is strong, but the more I let myself get carried away with such things the less likely I am to actually finish what I promised to do in the first place :) I’m also not too familiar with gitoxide’s codebase and haven’t fully wrapped my head around the code organization yet.

Some of these things like the missing API to set the fetch URL seem simple, though, so I’ll see if I find the time to send a PR for that. If not, maybe someone else will see the TODO comment and take up the mantle in future!

Thanks again for all your work on gitoxide and for your thoughts on my feedback; a Git reimplementation is a mammoth undertaking and I don’t think you have anything to be sorry for it not yet being perfectly polished! Now that we have code to shell out to git(1) for fetches and pushes, being able to drop our git2 dependency and rely on gitoxide (+ shelling out) alone is in sight. Hopefully that will help increase the synergy between Jujutsu and gitoxide development further :)

[yuja]

Looks good, thanks.

[bsdinis]

Are there any major blockers here at the moment? No pressure if you haven't found time for the last bits, but happy to help carry this through in any way I can

[bsdinis]

Thank you so much. Left a couple of nits and Qs, but I'd be fine with it being merged as is.
That being said, I think @yuja would have a better understanding here

[yuja]

LG, thanks!

[Byron]

It's great to see this PR merged, and I am sorry to probably have made the impression of being disinterested in what's going on it. That's not the case, fortunately, I just didn't have bandwidth and won't have bandwidth to do what I'd want to do: read each comment carefully and see if gitoxide documentation and/or APIs could be improved based on them.

So all I can do here is to encourage opening issues on gitoxide at least or PRs even while the pain is fresh, so there is a chance to eventually get the improvements that we deserve :).

Thanks and cheers

[emilazy]

Not at all, I appreciate you popping in :) We have some TODOs for things to upstream already, which I will try to get around to if time permits.