nixd icon indicating copy to clipboard operation
nixd copied to clipboard

vendor: enable written regression tests in derivations

Open Aleksanaa opened this issue 1 year ago • 3 comments

  • [x] #578 https://github.com/nix-community/nixd/blob/da2d7def2f9954aaf6cca72d84f4d7ebe3942caa/libnixf/default.nix#L33-L36
  • [ ] Missing doCheck = true in derivations.
  • [x] #541
  • [ ] Multi-threaded tests make nix sqlite busy, causing random failures in attrs-completion.md
******************** TEST 'nixd-attrset-eval :: attrs-completion.md' FAILED ********************
Script:
--
: 'RUN: at line 1';   nixd-attrset-eval --lit-test < /build/source/nixd/tools/nixd-attrset-eval/test/attrs-completion.md | FileCheck /build/source/nixd/tools/nixd-attrset-eval/test/attrs-completion.md
--
Exit Code: 2

Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "nixd-attrset-eval" "--lit-test"
# command stderr:
warning: '/nix/var/nix' does not exist, so Nix will use '/build/.local/share/nix/root' as a chroot store
terminate called after throwing an instance of 'nix::SQLiteBusy'
  what():  [31;1merror:[0m SQLite database '[35;1m/build/.local/share/nix/root/nix/var/nix/db/db.sqlite[0m' is busy

error: command failed with exit status: -6
$ "FileCheck" "/build/source/nixd/tools/nixd-attrset-eval/test/attrs-completion.md"
# command stderr:
FileCheck error: '<stdin>' is empty.
FileCheck command line:  FileCheck /build/source/nixd/tools/nixd-attrset-eval/test/attrs-completion.md

error: command failed with exit status: 2

--

********************
********************
Failed Tests (1):
  nixd-attrset-eval :: attrs-completion.md
  • [ ] 60s timeout is not actually enough for single-threaded builds, but sometimes enough (another source of random failures). https://github.com/nix-community/nixd/blob/da2d7def2f9954aaf6cca72d84f4d7ebe3942caa/nixd/tools/meson.build#L34
  • [x] #570

Aleksanaa avatar Jun 01 '24 19:06 Aleksanaa

The CI in this project never fails randomly.

https://github.com/nix-community/nixd/actions

60s timeout is not actually enough for single-threaded builds, but sometimes enough (another source of random failures).

Multi-threaded tests make nix sqlite busy, causing random failures in

Maybe just increase the testing time. But as you can see, it is always enough for github CI.

Regression testing nixpkgs-fmt. Instead, a script that outputs specific content should replace it, simply verifying that nixd passed the correct arguments to nixpkgs-fmt.

It won't increase the closure, right? Yes, a single script might be better.

inclyc avatar Jun 02 '24 01:06 inclyc

https://github.com/nix-community/nixd/blob/da2d7def2f9954aaf6cca72d84f4d7ebe3942caa/default.nix#L61

inclyc avatar Jun 02 '24 01:06 inclyc

Reopen as it is still planned. Generally we'd like to improve testing coverage in packaging environment. Thanks @Aleksanaa

inclyc avatar Jun 02 '24 05:06 inclyc