Draft: Support GPG signing #691

OdNairy · 2020-07-03T17:56:12Z

Closes: #42
GnuPG Made Easy (gpgme) documentation, source code
Completeness: 85%

Preview binary (use right click - Open to open the app):
GitUp.zip

Checklist (update: 23 Sep, 2022):

Technical debt:

Operations (amend, edit message, squash with parent) on single-commit repositories. Assert:

GitUp/GitUpKit/Core/GCCommitDatabase.m

Line 988 in 515ffe8

XLOG_DEBUG_CHECK(sqlite3_changes(_database) > 0);

Repository sample
Parse GPG signature in git_commit object for tests and GPG status presentation
git_commit_extract_signature

Raw commit data sample

tree a014809071940a9e860f33087275346130124b8f
parent 2a63c7ff507940895f4516f2d1f1c6d52a062b53
author Roman Gardukevich <[email protected]> 1603139410 +0300
committer Roman Gardukevich <[email protected]> 1603139580 +0300
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEL3huDZGHfHonBjlwEjx22Be1tNcFAl+N9/0ACgkQEjx22Be1
 tNcWAw//UFrd9DyIP4WnTZsVxmxeeUro6PCIRc4pMCaENxnmIAljHKAh0ofvuKci
 9Mj2aIxyffk5PsOPz0hoNT4jqm9EG8fYcyj+xJ2fGkGLDY+xXLRt44VPeBGoox3i
 Eqeb+SJCGBj7mhVtdAf9LjbexVPSfDHxPVciTKbks4IPN55xLbLebcJUSWRAXiPY
 gjCOAOQJiaaj16C5qEOhoh86k54BrzZK7ZCGI+U3o+YhMbhNj4qARnTdscl1lNDi
 udl3L5jfezWHFoYydpTBrH5StkWsROHjL0h/VKg4MqXRITXYyZlYJYR4yixlLHEf
 C+f1JEvj3mVqksMWf4SE0vkGQ3uudsF9kfHVAenhzBnl0GSy1NvLasrGMzeMqbYj
 iC5nlmr7schgpBDE3dCsW+9LGpcwlgPdZNyLphHkYayNtorM9G3SXH75ay2wj2F1
 MQXThsSsujVx+nSx5yVpcdf9ju8LflX+IARMXFMkV3HOh0rOK+JqyYascGuJ0zoZ
 GAnKLGt2umaN7kKG3Xvxdcsftxj4I85XTNJspaggLuIxdVuNvGghPjmmW9GlHYuA
 F+n8wyvHLnGa55X7ESkTH4759h8+GsE+/1kj3LcZWv25Av0bUGWOlZZ64aUofBTc
 k91ucZnutrVyevkVH9n02RRweM+Moibe40KvFe59WhhNX6RuXcI=
 =u7ec
 -----END PGP SIGNATURE-----
 

Hey!

Investigate manual calling gpg with pipelines in favor of gpgme dependency (git example)
Investigate replacement of gpgme_op_sign with gpgme_op_sign_start + gpgme_wait for async signature calculation. (doc1, doc2)

Useful links:

commit: add function to attach a signature to a commit libgit2/libgit2#3673 - commit: add function to attach a signature to a commit
Add sign capability to git_rebase_commit libgit2/libgit2#4913 - Add sign capability to git_rebase_commit
Introduce create_commit_cb, deprecate signing_cb libgit2/libgit2#6016

Technical references:

How does `git` sign commits?

Callstack:

How does `libgit2` sign commits?

Unfortunately, there are no overall sign commit support in libgit2. Also, GitUp uses it's own fork which is quite outdated. The good thing is that it gives us enough flexibility to implement custom signature adding functionality to libgit2 and reuse it in GitUpKit.

Dependencies

gitup/LibGit2 Pull Request: Support GPG libgit2#3

lucasderraugh · 2020-07-06T01:18:19Z

Thanks for contributing @OdNairy! This looks like a lot of works so thanks for pursuing it.

OdNairy · 2020-07-06T09:35:36Z

GitUpKit/Core/GCRepository+Bare.m

+  }
+
+  NSString* plainToSign = [[NSString alloc] initWithCString:body encoding:NSUTF8StringEncoding];
+  NSString* signature = [key signSignature:plainToSign];


@lucasderraugh I would like to trigger a separate discussion regarding UI blocking calls.
I'm focusing on signing commits functionality in current PR. However, even with the partial support I already observe significant UI regression – the main queue gets blocked for a 0.2-0.3 second when password for the gpg key already stored the keychain. It even can be an infinite period of time when GPG Keychain presents screen for GPG Key Password input.
Do you have any UI/UX related ideas on how we can play around this issue?

@lucasderraugh Any ideas?

Sorry, didn't realize there was an outstanding question. Let me think about what we could do here and I'll get back to you soon.

@lucasderraugh any updates? 🙂

OdNairy · 2020-07-06T09:49:39Z

GitUpKit/Core/GCRepository+Bare.m

+  }
+
+  if (gpgSignature != NULL) {
+    CALL_LIBGIT2_FUNCTION_GOTO(cleanupBuffer, git_commit_create_with_signature, &oid, self.private, commitBuffer.ptr, gpgSignature, NULL);


@lucasderraugh Separate thread regarding git-up/libgit2 fork.
It looks like libgit2 doesn't implement 100% gpg support yet so I have to implement some additional methods in it.
Is there any process for fork updates? Should it be just PR with functionality or I have to implement some tests in libgit2 as well?

Generally, what was the motivation to link against fork instead of upstream? Would it be possible to rebase on top of upstream? Is there a chance that we may migrate to the upstream instead of the fork?

/cc @pks-t @swisspol

I created a fork of libgit2 because a few patches were needed with some new APIs, bug fixes and change of behavior. Not all would be accepted in upstream and at the time maintainers could take weeks to respond. You can see the 15 or so custom commits here:
https://github.com/git-up/libgit2/commits/gitup

The README in the fork repo explains how to update to upstream.

@swisspol – looks like we standstill with libgit2, maybe suggested override below can help? (i think libgit2 are bit resistant to add features, even signing – they usually say it's more to the clients)

but gosh, I'm missing that in GitUp so much :) hope resolution will be found. Also, please, lmk if we can support somehow?

amatzen · 2020-12-02T10:21:19Z

So great! Can't wait to this being implemented. Thanks 🥇

nitz · 2021-08-24T12:30:43Z

Just started using GitUp this week and wondered where the slew of unsigned commits I had were coming from. The existence of this PR explains a lot 😅

Looks like you've put some serious work into this, @OdNairy, hugely appreciated that you decided too. Are you just waiting to hear back from @lucasderraugh on the UI issue?

I'd like to offer my two cents on the topic, if it helps. I use TortoiseGit as my non-terminal interface to Git when I'm on Windows. Combined with Gpg4win, I'm able to sign my commits there. TortoiseGit doesn't actually do the git operations itself, just invokes the standard binary with specific options. That means that when it comes time to commit, git itself respects commit.gpgsign, invoking gpg.program all by itself. It's nice that it just respects my config, but the interop between those layers is slow to say the least. My GPG PK is stored on a smartcard, so the first commit I've made since inserting the card will force gpg-agent to fire up pinentry to unlock the smartcard. Depending on what else my machine is doing at the time, this process takes an agonizingly long time (especially if Gpg4win is firing up it's whole setup because it wasn't already running.) This has had the effect of causing my commit to time out and fail, because it gets tired of waiting.

I tell this ridiculous story to illustrate that the issue you've run into isn't abnormal, and at least in my case is something I'd absolutely expect. A sub half-second hit while it grabs the key from the agent isn't something I'd even blink at. Further, allowing a reasonable wait (20 secs +/-?) for pinentry to complete, and failing if it doesn't is as well par for the course for me. I've long ago accepted that when dealing with gpg, things tend to be a little more "janky" than one might expect, and am personally fully comfortable with that fact by this point.

Cheers

lucasderraugh · 2021-08-24T14:44:36Z

@nitz I don't have tons of time to review the details of PRs these days. But I'm still willing to help move the project forward.

I honestly don't care what the initial UI looks like (UI can easily be iterated on in future releases), but here is the 1 thing I do care about:

Adding in support for GPG signing CANNOT regress the performance or workflows for people not using this feature.

I'm not sure if @OdNairy is willing to keep working on these changes, I personally don't have a good understanding of gpg signing so I'm deferring to those that do to understand a good approach here. I will absolutely test out the final result before merging in anything, but that's where I stand here.

OdNairy · 2021-08-24T15:54:24Z

Adding in support for GPG signing CANNOT regress the performance or workflows for people not using this feature.

Definitely makes sense. That implementation doesn't impact performance for non-gpg users. Only GitUp binary size would be impacted for everybody.

robertlacroix · 2022-02-09T23:18:58Z

@OdNairy this PR looks great! As you probably know Git shells out to gpg, which enabled other implementations, like an x509 one github/smimesign and others. While using a library is obviously better than executing a binary, the downside is that those alternatives don't work out of the box.

Have you considered simplifying this, by just supporting what's configured in gitconfig and executing the configured gpg.program like Git does to handle the signing?

OdNairy · 2022-09-22T07:38:43Z

Hi all
I have some news – after becoming a dad of twins I'm back to the gpg support 😄
Thanks everybody for your support and comments, it what was motivated me to get back to work.

There are few updates:

Rebased on top of the latest GitUp and libgit2 fork
Script to build gpgme library as XCFramework artefact
Apple Silicon support

I also can confirm that all operations still works as expected. Except the single-commit repository 🤡 . There is something broken with that thing so I probably need help from @lucasderraugh with it.

Stay tuned!

GitUpKit/Core/GCRepository+HEAD.m

GitUpKit/Core/GCRepository+Bare.m

The System Region leads to false positive test failures in some regions

Supported operations: * Commit * Merge * Cherry-pick

OdNairy · 2022-09-30T20:51:23Z

Update: I was able to eliminate almost all extra code from the libgit2 to support GPG sign. It is a single new function PR now 🎉 git-up/libgit2#3.
Next step is to start writing some unit tests.

OdNairy · 2022-10-16T11:31:57Z

I've found a serious issue in my PR. When GitUp overwrite commit of some other author, my implementation still signed it with local key. The git doesn't do it. So I have to implement some filtering logic first.

gelosi · 2023-05-28T11:13:12Z

@OdNairy hi! Any chances to move this rock forward? 👋

OdNairy force-pushed the feature/gpg branch 2 times, most recently from 8f40793 to 3d5292e Compare July 3, 2020 19:05

OdNairy commented Jul 6, 2020

View reviewed changes

OdNairy force-pushed the feature/gpg branch from d5a8d38 to 5211f8f Compare October 19, 2020 19:58

OdNairy force-pushed the feature/gpg branch from 0eb80f0 to a152b20 Compare September 22, 2022 07:32

osxd reviewed Sep 22, 2022

View reviewed changes

GitUpKit/Core/GCRepository+HEAD.m Outdated Show resolved Hide resolved

OdNairy force-pushed the feature/gpg branch 2 times, most recently from 10aa199 to af7370c Compare September 23, 2022 11:38

osxd reviewed Sep 23, 2022

View reviewed changes

GitUpKit/Core/GCRepository+Bare.m Outdated Show resolved Hide resolved

OdNairy added 4 commits September 26, 2022 13:08

Xcode autoupdate: GitUp.xcodeproj format

9006c48

Use US region for tests

7c2f656

The System Region leads to false positive test failures in some regions

Add GPGMe library to work with gpg agent

0d3b71b

Basic commit GPG sign

7e17d4e

Supported operations: * Commit * Merge * Cherry-pick

OdNairy force-pushed the feature/gpg branch 2 times, most recently from 96d073e to ac1ce50 Compare September 30, 2022 20:45

Sign commits in advanced operations

79b7591

OdNairy force-pushed the feature/gpg branch from ac1ce50 to 79b7591 Compare October 1, 2022 08:58

Merge branch 'master' into feature/gpg

b9fb4d2

lucasderraugh mentioned this pull request Dec 1, 2022

Add support for the new includeIf directive #315

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Draft: Support GPG signing #691

Draft: Support GPG signing #691

OdNairy commented Jul 3, 2020 •

edited

Loading

lucasderraugh commented Jul 6, 2020

OdNairy Jul 6, 2020

OdNairy Oct 20, 2020

lucasderraugh Oct 21, 2020

MrSkwiggs Nov 26, 2020

OdNairy Jul 6, 2020

swisspol Jul 6, 2020

osxd Sep 19, 2022

amatzen commented Dec 2, 2020

nitz commented Aug 24, 2021

lucasderraugh commented Aug 24, 2021 •

edited

Loading

OdNairy commented Aug 24, 2021

robertlacroix commented Feb 9, 2022

OdNairy commented Sep 22, 2022 •

edited

Loading

OdNairy commented Sep 30, 2022

OdNairy commented Oct 16, 2022

gelosi commented May 28, 2023

Draft: Support GPG signing #691

Are you sure you want to change the base?

Draft: Support GPG signing #691

Conversation

OdNairy commented Jul 3, 2020 • edited Loading

Checklist (update: 23 Sep, 2022):

Technical debt:

Useful links:

Technical references:

How does git sign commits?

How does libgit2 sign commits?

Dependencies

lucasderraugh commented Jul 6, 2020

OdNairy Jul 6, 2020

Choose a reason for hiding this comment

OdNairy Oct 20, 2020

Choose a reason for hiding this comment

lucasderraugh Oct 21, 2020

Choose a reason for hiding this comment

MrSkwiggs Nov 26, 2020

Choose a reason for hiding this comment

OdNairy Jul 6, 2020

Choose a reason for hiding this comment

swisspol Jul 6, 2020

Choose a reason for hiding this comment

osxd Sep 19, 2022

Choose a reason for hiding this comment

amatzen commented Dec 2, 2020

nitz commented Aug 24, 2021

lucasderraugh commented Aug 24, 2021 • edited Loading

OdNairy commented Aug 24, 2021

robertlacroix commented Feb 9, 2022

OdNairy commented Sep 22, 2022 • edited Loading

OdNairy commented Sep 30, 2022

OdNairy commented Oct 16, 2022

gelosi commented May 28, 2023

OdNairy commented Jul 3, 2020 •

edited

Loading

How does `git` sign commits?

How does `libgit2` sign commits?

lucasderraugh commented Aug 24, 2021 •

edited

Loading

OdNairy commented Sep 22, 2022 •

edited

Loading