Feature requests for puppeting Hydra from other CI systems

[mikepurvis]

Hello there! My company is on a Nix journey and part of this has been figuring out how Hydra fits into our CI strategy (which is otherwise comprised of Jenkins and GitLab CI pipelines). Overall, we are finding that the Hydra fundamentals are really good, but the frontend isn't a good fit for our needs:

• The projects/jobsets/evals setup gets in the way of the workflow we want. We use flakes exclusively, and currently every new flake tag ends up created as its own jobset, which means that there's only ever a single evaluation of each (since we can't trigger an eval for a particular tag—it's always just "take the latest").
• There's no API for eval progress information, buildstep breakdown, or logs.
• Sometimes it's not convenient to have to explicitly cancel an eval. It would be great to have an option for an API-triggered eval where it relies on an external keepalive, and if that keepalive disappears, it self-cancels.

What we'd really like is to be able to just trigger evals of arbitrary flakes— further bonus if we can upload a flake tarball directly to Hydra over the API rather than it needing to be hosted on a URL or git repo somewhere.

So the question is, is this functionality that Hydra would have any interest in offering, or is it just too far out of scope for Hydra's intended use cases? If out of scope, would it be something that could be reasonable to build ourselves using portions of Hydra? Or would it make more sense to build a whole new thing from scratch and just directly call the nix CLI?

Thanks for any thoughts or advice, definitely looking forward to potentially collaborating!

[Ericson2314]

I would like Hydra to act as a remote builder, such that one can just submit ad-hoc jobs from the local CLI.

It sounds like that is something that might help you out too.

[nlewo]

I'm also interested in such feature (and really like how Hydra is efficient!).

Currently, i already use Hydra and report status in GitLab/GitHub CI:

1. the repository contains a declarative Hydra jobset with the GitLab plugin: each minute, Hydra polls the repository and creates a jobset per pull request (i guess GitLab could use the Hydra API to ask for an eval in order to avoid this 1 minute period)
2. GitLab runs a job and executes hydra-cli jobset-wait: this polls the Hydra API to get the pull request jobset status and report it once finished.

Ok, I confess this is a bit hacky! But it works quite ok in practice.

A first nice and short term improvement would be a have an Hydra API route to get the status of a jobset: existence, latest evaluated inputs, evaluation status, builds status (this is currently implemened in the hydra-cli tool).

[mikepurvis]

@Ericson2314 What would Hydra-from-the-CLI give you that you can't get from just running ordinary nix build commands from the CLI? Is it just linkable webpages for build status and logs? I'm wondering if there's room here for a project that basically just provides a dirt-simple HTTP interface to nix log, nix realisation info, nix path-info, and friends? It may well be that nix-serve is most of the way to this, and it would only be a thin bit of chrome needed on top?

@nlewo Sounds like we're doing some similar stuff. One of the big things for my org is that we don't at all have a 1:1 mapping between the CI builds we want/need and the constituent source repositories. It's many (hundreds) of repos feeding into a single large build process, and part of why the "other" CI system has to exist to wrap Hydra is to run that generator step, gather up all the narhashes for everything, etc.

I think the biggest thing for me in terms of the architecture of Hydra really would be divorcing evaluations from jobsets— make locked flake evaluations the plumbing, with a rich API for creation, monitoring, lifecycle, etc, and make projects/jobsets an optional porcelain on top for users who don't want to roll their own.