gpg-agent: darwin support via launchd service

[montchr]

Description

Add support for running the gpg-agent service on Darwin/macOS via launchd.

Additional Context

Retains the existing gtk default for cfg.pinentryFlavor which works on macOS, although in my experience the dialog doesn't focus automatically. I suspect the focus issue is caused by yabai (I've also had issues with yabai not managing new emacs frames).

I had initially tried introducing support for pinentry-mac, but the version available in Nixpkgs is years out of date, and it doesn't fit neatly within the 'flavor' concept since it's an entirely different package than the usual pinentry.

Checklist

• [x] Change is backwards compatible.

• [x] Code formatted with ./format.

• [x] Code tested through nix-shell --pure tests -A run.all.

• [x] Test cases updated/added. See example.

• [x] Commit messages are formatted like

  {component}: {description}
  
  {long description}
  

  See CONTRIBUTING for more information and recent commit messages for examples.

• If this PR adds a new module

  • [ ] Added myself as module maintainer. See example.

  • [ ] Added myself and the module files to .github/CODEOWNERS.

[rycee]

Thanks for the contribution! I've added a few comment 🙂

[montchr]

Update: I don't have access to a system running macOS at the moment (NixOS only right now), but I expect I will be able to follow up on this in the next couple weeks.

[montchr]

I think this is ready for another review, though I should note that I would prefer that it not be merged until someone can verify it on macOS. I expected that I'd be able to do so by this point, but it'll be a bit longer before I have a Mac to test on. In the meantime, I'll keep tracking this version of the module on NixOS (but it's been working fine there for a while now).

Also, the new tests pass, however some unrelated test failed:

building '/nix/store/8xjprr5yi0r9j8j8b6p2nh3rn7jykdrm-neovim-0.7.0.drv'...
Generating remote plugin manifest
Error detected while processing function coc#rpc#start_server[13]..coc#util#get_data_home:
line   15:
E117: Unknown function: coc#float#create_notification
remote/host: generated rplugin manifest: /nix/store/8384rxk99idi5pz20hf233fdp69i5axi-neovim-0.7.0/rplugin.vim
Generating rplugin.vim failed!
error: builder for '/nix/store/8xjprr5yi0r9j8j8b6p2nh3rn7jykdrm-neovim-0.7.0.drv' failed with exit code 1;
       last 6 log lines:
       > Generating remote plugin manifest
       > Error detected while processing function coc#rpc#start_server[13]..coc#util#get_data_home:
       > line   15:
       > E117: Unknown function: coc#float#create_notification
       > remote/host: generated rplugin manifest: /nix/store/8384rxk99idi5pz20hf233fdp69i5axi-neovim-0.7.0/rplugin.vim
       > Generating rplugin.vim failed!
       For full logs, run 'nix log /nix/store/8xjprr5yi0r9j8j8b6p2nh3rn7jykdrm-neovim-0.7.0.drv'.
error: 1 dependencies of derivation '/nix/store/pmslzabl2mh96hyrmranmsnvg87s2ahv-home-manager-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/4dr1jnnlbpxsl3xq7bx963zk74vxr1rv-home-manager-generation.drv' failed to build
error: 1 dependencies of derivation '/nix/store/0ih32vl40zpck5vjjsw5vpfifg76m252-nmt-report-neovim-coc-config.drv' failed to build
(use '--show-trace' to show detailed location information)

[treed]

I attempted to test this on my Mac by applying the patch over master. I did have to edit the patch to change context in the tests file (wezterm was added), but other than that it applied cleanly.

It didn't immediately work and I was unable to figure out where launchd keeps its logs. One page I found mentioned /var/log/system.log but nothing about this was there. I had to add the following to get any logging output:

    launchd.agents.gpg-agent.config = {
      StandardErrorPath = "/tmp/gpg-agent.err.log";
      StandardOutPath = "/tmp/gpg-agent.out.log";
    };

(This seems relevant to discussion I've seen elsewhere about standard logging paths.)

Stderr consists entirely of lines like the following:

gpg-agent[19259]: no gpg-agent running in this session

Which strikes me as sort of circular, but maybe someone with more familiarity with gpg-agent will know what the problem might be.

[treed]

After a bit of digging around in gpg-agent source code and some searching, I figured out that the problem is that launchd does not automatically include the binary as arg 0: https://apple.stackexchange.com/questions/110644/getting-launchd-to-read-program-arguments-correctly

I then corrected the config to:

            ProgramArguments =
              [ "${gpgPkg}/bin/gpg-agent" "--supervised" (mkIf cfg.verbose "--verbose") ];

Which gave me a different error message:

gpg-agent[21550]: WARNING: "--supervised" is a deprecated option
gpg-agent (GnuPG) 2.3.6 starting in supervised mode.
no LISTEN_PID environment variable found in --supervised mode (ignoring)
no LISTEN_FDS or LISTEN_FDNAMES environment variables found in --supervised mode (assuming 1 active descriptor)
Fatal: file descriptor 3 must be valid in --supervised mode if LISTEN_FDNAMES is not set

I'm unsure how to get the socket through as those environment variables. I'm also curious what we're supposed to use other than --supervised if that's deprecated. It looks like they just deprecated it without replacement: https://dev.gnupg.org/rGca5d5142c6d6eaba4572a086f8473e4aebdd3f9e

[treed]

I will probably leave this alone for now, but at the very least I think there's something wrong with the Socket options. I don't see anything with /tmp/gnupg existing while this service is being restarted, even if I watch with a while true loop to see if it's being deleted immediately as the service fails.

Also, I think there's an error with the mode, which should be converted from octal to decimal per the docstring (384 in the case of 600 octal), but changing this doesn't seem to solve anything.

[stale[bot]]

Thank you for your contribution! I marked this pull request as stale due to inactivity. Please read the relevant sections below before commenting.

If you are the original author of the PR

• GitHub sometimes doesn't notify people who commented / reviewed a PR previously when you (force) push commits. If you have addressed the reviews you can officially ask for a review from those who commented to you or anyone else.
• If it is unfinished but you plan to finish it, please mark it as a draft.
• If you don't expect to work on it any time soon, please consider closing it with a short comment encouraging someone else to pick up your work.
• To get things rolling again, rebase the PR against the target branch and address valid comments.

If you are not the original author of the PR

• If you want to pick up the work on this PR, please create a new PR and indicate that it supercedes and closes this PR.

[montchr]

I've rebased the branch onto the latest from master.

I also noticed #3183 which addressed my opening note about pinentry_mac. That branch merges into this one cleanly. I've pushed the combined changes to a new branch on my fork which I am now tracking in my personal configs.

https://github.com/montchr/home-manager/compare/gpg-agent-darwin...montchr:home-manager:gpg-agent-darwin-with-pinentry-mac?expand=1

https://github.com/montchr/dotfield/commit/d9ca7ac828b2b4b39ab336aed00ab093f89bb083

Commit signing has been working for me with the default pinentry flavor (though a sub-optimal UX). With #3183 in the combined branch, I'm now using pinentry_mac.

I haven't been using the SSH agent functionality lately, so I'll re-enable that and take a look at some of your suggestions @treed

[montchr]

I've updated this to include the changes from #3183 and as discussed in that PR I've also updated the default pinentry program to pinentry-mac on macOS. I figured that since this change is specific to the graphical macOS as opposed to the underlying Darwin system, Darwin still will use the gtk2 default.

[montchr]

I'm wondering if this -z logic might not work on Darwin:

https://github.com/nix-community/home-manager/blob/6acfdecd72cf81f27556a8522f8dd752e7f070a7/modules/services/gpg-agent.nix#L242-L246

FWIW nix-darwin does not do such a check, and I've confirmed that it works as expected:

https://github.com/LnL7/nix-darwin/blob/6349b99bc2b96ded34d068a88c7c5ced406b7f7f/modules/programs/gnupg.nix#L43-L49

And running this in an active shell session allows me to use gpg-agent for SSH in that session:

home-manager on  gpg-agent-darwin
❯ export SSH_AUTH_SOCK="$(gpgconf --list-dirs agent-ssh-socket)"

home-manager on  gpg-agent-darwin
❯ ssh-add -L
ssh-rsa <stuff...> cardno:xx xxx xxx

But in a new shell session I get this:

❯ echo $SSH_AUTH_SOCK
/private/tmp/com.apple.launchd.5gcAGrtCqp/Listeners

[montchr]

I will probably leave this alone for now, but at the very least I think there's something wrong with the Socket options.

@treed Yeah it's probably not worth pursuing sockets via launchd for this right now (if ever). Do the current changes work for you?

[sikmir]

What about pinentry-touchid support? It would be really cool!

[cmacrae]

Hey all 👋 Hope people don't mind me ressurecting this PR. Thanks for putting in the groundwork @montchr, nice to see you again :) I cooked up my own PR which was unknowingly, in large part, a repetition of these changes. Though you implemented some things nicer than I had!

I have a version of this that I'm using on my system (macOS Ventura 13.3, aarch64-darwin) with gpg-agent & ssh-agent functionality working flawlessly. It also considers some updates that were recently introduced to home-manager in which we no longer need to do so much control logic based on stdenv.hostPlatform (see #2915 ).

It appears #3840 was opened just the other day, but I don't think it addresses all the edgecases considered in changes here.

I'd love to see this accepted upstream, so what's the best path forward? A PR into your branch?

Based on their past involvement/being listed as reviewers, courtesy ping for @ratorx & @rycee

[cmacrae]

What about pinentry-touchid support? It would be really cool!

@sikmir sure :) If that fits your security posture, you could easily configure this module to use it with something like:

# Explicitly set to `null` as we infer "mac" if `stdenv.hostPlatform.isDarwin` is true
programs.gpg-agent.pinentryFlavor = null;
programs.gpg-agent.exraConfig = ''
  # or wherver Homebrew put pinentry-touchid
  pinentry-program /usr/local/opt/pinentry-touchid/bin/pinentry-touchid
'';

[montchr]

Hey @cmacrae! Thanks for resurfacing this. It certainly stagnated but I've been using it in my own configs for a while since I've been super busy the past few months and dealing with the oddities of macOS can be… frustrating. So if you (or anyone else) has made some progress, a PR into this branch would be great. ~Though now that there's three separate PRs I'm not sure which would be the best path forward?~

[cmacrae]

@montchr Thanks for getting back :) Glad it's been working nicely for you. Understandable, we're all living our own busy lives! I've opened a PR to your branch with all the changes I've mentioned here. Once that's rebased in, we should be in a good position to get this merged. I updated the test cases and have confirmed all tests pass.

[montchr]

I've just rebased onto master with @cmacrae's additions. I'm now tracking the latest commit in my personal configuration and git commit signing on aarch64-darwin is working, just as it was before.

However, some of the new tests are failing:

gpg-agent-default-homedir: FAILED
For further reference please introspect /nix/store/y5lp84fzwq499bj5sl4abny9idyl6vys-nmt-report-gpg-agent-default-homedir
gpg-agent-override-homedir: FAILED
For further reference please introspect /nix/store/358ql8ibzmn6jm7qkdgg9xsggkiprval-nmt-report-gpg-agent-override-homedir
gpg-immutable-keyfiles: OK
gpg-mutable-keyfiles: OK
gpg-override-defaults: OK

Not quite sure what to make of the failure even after poking around in the store paths mentioned in the error output. @cmacrae any insights?

[cmacrae]

Not quite sure what to make of the failure even after poking around in the store paths mentioned in the error output. @cmacrae any insights?

Oh... weird 🤔 I ran the tests locally and they worked fine (as you can see in dd69b63, I made changes to make sure they worked). I can take a look when I get a moment

[jaudiger]

I was looking for gpg-agent on darwin, and found this PR. I just wanted to express my interest for this ongoing work 🌟

If I can help to test this PR, don't hesitate to ping me @montchr

[stale[bot]]

Thank you for your contribution! I marked this pull request as stale due to inactivity. Please read the relevant sections below before commenting.

If you are the original author of the PR

• GitHub sometimes doesn't notify people who commented / reviewed a PR previously when you (force) push commits. If you have addressed the reviews you can officially ask for a review from those who commented to you or anyone else.
• If it is unfinished but you plan to finish it, please mark it as a draft.
• If you don't expect to work on it any time soon, please consider closing it with a short comment encouraging someone else to pick up your work.
• To get things rolling again, rebase the PR against the target branch and address valid comments.

If you are not the original author of the PR

• If you want to pick up the work on this PR, please create a new PR and indicate that it supercedes and closes this PR.