Hercules-effects support

[Mic92]

Started in https://github.com/Mic92/buildbot-nix/pull/68

[zowoq]

@Mic92 How you you plan to handle secrets? Separate secrets file for each repo or just one secret file per instance like hercules?

[Mic92]

Ideally one per repository if I can make it secure. Maybe each repository gets a public key for encryption.

[zowoq]

I forgot to ask about the GitToken secret, it is a short lived token from the github app set by the hercules backend.

Possible that some repos may not even need to set secrets if they had this token as it could be used for pushing to the repo, pages, releases, etc.

[zowoq]

https://github.com/hercules-ci/hercules-ci-agent/blob/master/docs/modules/ROOT/pages/effects.adoc The store and a nix-daemon socket are accessible.

There is some logic for the socket in hercules-ci-agent/src/Hercules/Effect.hs, not sure how much of that would need to be replicated.

[zowoq]

These are the changes that were needed for a hci-effects.runNixDarwin deployment to work. I guess this would be okay while buildbot effects are restricted to running on the default branch?

https://buildbot.nix-community.org/#/builders/3618/builds/24/steps/3/logs/stdio

diff --git a/buildbot_effects/buildbot_effects/__init__.py b/buildbot_effects/buildbot_effects/__init__.py
index 7b2273e90..c60aa5dcc 100644
--- a/buildbot_effects/buildbot_effects/__init__.py
+++ b/buildbot_effects/buildbot_effects/__init__.py
@@ -190,6 +190,7 @@ def run_effects(
     env["HERCULES_CI_SECRETS_JSON"] = "/run/secrets.json"
     env["NIX_BUILD_TOP"] = "/build"
     env["TMPDIR"] = "/tmp"  # noqa: S108
+    env["NIX_REMOTE"] = "daemon"
     clear_env = set()
     clear_env.add("TMP")
     clear_env.add("TEMP")
@@ -235,6 +236,9 @@ def run_effects(
         "/nix/store",
         "--hostname",
         "hercules-ci",
+        "--bind",
+        "/nix/var/nix/daemon-socket/socket",
+        "/nix/var/nix/daemon-socket/socket",
     ]
 
     with NamedTemporaryFile() as tmp:

[Mic92]

I forgot to ask about the GitToken secret, it is a short lived token from the github app set by the hercules backend.

Possible that some repos may not even need to set secrets if they had this token as it could be used for pushing to the repo, pages, releases, etc.

Should be easy to generate in buildbot as well. Is this provided unconditionally?

[Mic92]

These are the changes that were needed for a hci-effects.runNixDarwin deployment to work. I guess this would be okay while buildbot effects are restricted to running on the default branch?

https://buildbot.nix-community.org/#/builders/3618/builds/24/steps/3/logs/stdio

diff --git a/buildbot_effects/buildbot_effects/init.py b/buildbot_effects/buildbot_effects/init.py index 7b2273e90..c60aa5dcc 100644 --- a/buildbot_effects/buildbot_effects/init.py +++ b/buildbot_effects/buildbot_effects/init.py @@ -190,6 +190,7 @@ def run_effects( env["HERCULES_CI_SECRETS_JSON"] = "/run/secrets.json" env["NIX_BUILD_TOP"] = "/build" env["TMPDIR"] = "/tmp" # noqa: S108

• env["NIX_REMOTE"] = "daemon" clear_env = set() clear_env.add("TMP") clear_env.add("TEMP") @@ -235,6 +236,9 @@ def run_effects( "/nix/store", "--hostname", "hercules-ci",

•    "--bind",
  

•    "/nix/var/nix/daemon-socket/socket",
  

•    "/nix/var/nix/daemon-socket/socket",
  

  ]

  with NamedTemporaryFile() as tmp:

We already provide this type of access to CI users anyway. The only thing we should not do is giving it trusted user access.

[zowoq]

Is this provided unconditionally?

No, not in the base effect, would need to set secretsMap.token = { type = "GitToken"; };. It is set by default for a few of the specialised effects in https://github.com/hercules-ci/hercules-ci-effects, e.g. publishing releases, pushing branches.