Nim icon indicating copy to clipboard operation
Nim copied to clipboard

Published 20 hours ago • verified publisher icon nim-lang

Package CI candidates

Open metagn opened this issue 1 year ago • 2 comments

This is a general issue for packages that should be in package CI until the situation has sufficiently improved, suggestions would be appreciated.

The release of 2.2 was delayed quite a bit by breaks in packages that were not tested in CI and had to be tested individually long after the breaks were introduced. The last time a package was added to CI was constantine in July 2024, which is somewhat recent, but the last ones before that were ssostrings and cowstrings in June 2023, before the release of 2.0. I can attest to constantine for example being the only package in CI which demonstrated some breaks when developing PRs like #24005. So it's long overdue for some updates.

Status

Individual libraries

Going off the list of packages which @narimiran made PRs to add 2.2 CI, not all are necessarily feasible for Nim CI:

Other status packages that may be worth adding:

Except ssz-serialization, all of these have CI runs that last around 10 minutes at most (some don't have CI runs though). In the worst case, we don't have to run their entire test suites (as is done for stint).

Big packages

Most of the above libraries are dependencies of the bigger status toolchain repos, namely:

These are the packages that catch the most regressions by far. It might not be necessary to test all of the ones above that are dependencies to these if we just test these.

These packages are quite a bit more difficult to test however, both due to the required setup and maintenance, as well as the potential increase to the CI time, since the full CI runs of these are 30 minutes or longer. Some options we have to deal with this are:

  • Add a dedicated status batch similar to allowed_failures
  • Don't run this batch for every single commit/PR: either allow manually triggering it for particularly impactful PRs, or run it on a schedule, like once a week. Or both.
  • Do it in another repo similar to nightlies, ideally with support for CI'ing specific PRs to this repo.

This could be generalized to test any large toolchain using Nim.

Unfortunately I don't know much about the CI for these packages so I can't really go into specifics. Other opinions on this would be appreciated.

nim-lang packages

These have the advantage that we can manage & watch their CI, but it might still be useful to test them for PRs etc.

  • nimble, compiler does not use HEAD version and so the HEAD version isn't tested, has own devel CI though
  • nimlangserver, nimlsp is tested but langserver is on the nim-lang org and more actively developed, has own devel CI, caught #24094
    • see https://github.com/nim-lang/Nim/issues/24266#issuecomment-2405246714
  • threading, has own devel CI

Other community packages

Unfortunately again, as someone who mainly uses Nim to work on the Nim compiler, I'm not too aware of libraries that are important to people that don't interact with this repo. Off the top of my head, I can suggest:

  • nimja or other template libraries, currently only templates is tested
  • arturo which recently encountered #23587, maybe min, spry if their code is not too old
  • maybe chame, got #24288 although this wouldn't be caught by testing

Potentially harder to test:

  • newer graphics libraries like owlkettle (GTK4 dependency, there is currently a problem linking to libraries in macos CI since #24157), naylib
  • newer server libraries in jester/httpbeast's abscence, like mummy; currently prologue is tested

Not all tested packages need to be popularly used, as long as they uniquely demonstrate a use of the language that is expected to keep working. Small, vetted macro libraries for example would be useful.

metagn avatar Oct 09 '24 10:10 metagn