alphapapa
A test possibly failing on Org 9.4
Hello,
I'm observing the failures below when building 0.5 on GNU Guix, with the following dependency versions:
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[...]
Org link safety
buffers-files parameter
Errors for a quoted lambda (0.99ms)
Errors for an unquoted lambda (0.85ms)
Errors for a quoted lambda in a list (1.31ms)
Errors for an unquoted lambda in a list (0.85ms)
super-groups parameter
Errors for a quoted lambda (2.75ms)
Errors for an unquoted lambda (17.72ms)
Errors for a quoted expression (2.94ms)
Errors for an unquoted expression (2.63ms)
Errors for a :pred group Expected `(open-link pred-selector-link)' to throw a child signal of `error' with args `("Unsafe groups disallowed (:pred): (lambda (_) (error UNSAFE))")', but instead it threw (error "UNSAFE") (2.68ms)
Errors for an :auto-map group Expected `(open-link auto-map-selector-link)' to throw a child signal of `error' with args `("Unsafe groups disallowed (:auto-map): ((lambda (_) (error UNSAFE)))")', but instead it threw (error "UNSAFE") (2.67ms)
title parameter
Errors for a quoted lambda (2.53ms)
Errors for an unquoted lambda (2.51ms)
Errors for an expression (17.77ms)
sort parameter
Errors for a quoted lambda (0.87ms)
Errors for an unquoted lambda (0.84ms)
Errors for a quoted lambda in a list (1.31ms)
Errors for an unquoted lambda in a list (0.83ms)
View saving/loading
Bookmarks
Grouping
is restored (10.13ms)
Queries
Sexps match (22.18ms)
Strings match (6.94ms)
Sorting
One sorter is restored (21.70ms)
Multiple sorters are restored (6.66ms)
Buffers/Files
One filename matches (4.93ms)
A list of filenames matches (22.85ms)
One buffer matches (4.90ms)
A list of buffers matches (7.57ms)
Dynamic blocks
warn about sexp queries
when org-ql-ask-unsafe-queries is non-nilUpdating dynamic block ‘org-ql’ at line 3...
when org-ql-ask-unsafe-queries is non-nil (20.37ms)
unless org-ql-ask-unsafe-queries is nilUpdating dynamic block ‘org-ql’ at line 3...
Updating dynamic block ‘org-ql’ at line 3...done
unless org-ql-ask-unsafe-queries is nil (8.19ms)
Links
Queries
in sexp form
prompt when `org-ql-ask-unsafe-queries' is non-nil
and signal an error when rejected by user (13.23ms)
and run when approved by user (29.47ms)
don't prompt when `org-ql-ask-unsafe-queries' is nil (13.61ms)
match after restoring (28.73ms)
in raw sexp form
prompt when `org-ql-ask-unsafe-queries' is non-nil
and signal an error when rejected by user (5.77ms)
and run when approved by user (7.14ms)
don't prompt when `org-ql-ask-unsafe-queries' is nil (7.08ms)
match after restoring (22.12ms)
could be evil when not prompted about (7.29ms)
in string form
match after restoring (14.22ms)
Grouping
is restored (29.31ms)
Sorting
One sorter is restored (13.78ms)
Multiple sorters are restored (30.69ms)
Buffers/Files
Can search a file by filename (13.51ms)
Can search multiple files by filename (30.89ms)
Can search buffer containing the link Expected `(var-after-link-save-open 'org-ql-view-buffers-files one-filename query :buffer link-buffer)' to be `equal' to `#<buffer *TEST LINK BUFFER*>', but instead it was `("/tmp/guix-build-emacs-org-ql-0.5.drv-0/test-org-ql-U1M4Pv/test1.org" "/tmp/guix-build-emacs-org-ql-0.5.drv-0/test-org-ql-U1M4Pv/test2.org")' which does not match because: (different-types ("/tmp/guix-build-emacs-org-ql-0.5.drv-0/test-org-ql-U1M4Pv/test1.org" "/tmp/guix-build-emacs-org-ql-0.5.drv-0/test-org-ql-U1M4Pv/test2.org") #<buffer *TEST LINK BUFFER*>). (30.06ms)
========================================
org-ql Org link safety super-groups parameter Errors for a :pred group
Traceback (most recent call last):
(buttercup-fail "%s" "Expected `(open-link pred-selector-link)' to throw a...
(signal buttercup-failed "Expected `(open-link pred-selector-link)' to thr...
FAILED: Expected `(open-link pred-selector-link)' to throw a child signal of `error' with args `("Unsafe groups disallowed (:pred): (lambda (_) (error UNSAFE))")', but instead it threw (error "UNSAFE")
========================================
org-ql Org link safety super-groups parameter Errors for an :auto-map group
Traceback (most recent call last):
(buttercup-fail "%s" "Expected `(open-link auto-map-selector-link)' to thr...
(signal buttercup-failed "Expected `(open-link auto-map-selector-link)' to...
FAILED: Expected `(open-link auto-map-selector-link)' to throw a child signal of `error' with args `("Unsafe groups disallowed (:auto-map): ((lambda (_) (error UNSAFE)))")', but instead it threw (error "UNSAFE")
========================================
org-ql View saving/loading Links Buffers/Files Can search buffer containing the link
Traceback (most recent call last):
(buttercup-fail "%s" "Expected `(var-after-link-save-open 'org-ql-view-buf...
(signal buttercup-failed "Expected `(var-after-link-save-open 'org-ql-view...
FAILED: Expected `(var-after-link-save-open 'org-ql-view-buffers-files one-filename query :buffer link-buffer)' to be `equal' to `#<buffer *TEST LINK BUFFER*>', but instead it was `("/tmp/guix-build-emacs-org-ql-0.5.drv-0/test-org-ql-U1M4Pv/test1.org" "/tmp/guix-build-emacs-org-ql-0.5.drv-0/test-org-ql-U1M4Pv/test2.org")' which does not match because: (different-types ("/tmp/guix-build-emacs-org-ql-0.5.drv-0/test-org-ql-U1M4Pv/test1.org" "/tmp/guix-build-emacs-org-ql-0.5.drv-0/test-org-ql-U1M4Pv/test2.org") #<buffer *TEST LINK BUFFER*>).
Ran 308 specs, 3 failed, in 2.10s.
After 0 kbd macro iterations:
command "buttercup" "-L" "." failed with status 255
I've disabled the three failing tests for now.
Thanks!
Hello,
Please do not disable the tests, especially the ones that fail saying UNSAFE. It says UNSAFE because it indicates that the code being tested is failing in an unsafe way, which could be a security vulnerability.
You listed the package versions, but what about the Emacs version?
Also, to be clear, are you packaging this for Guix officially, or is this just for your personal use?
Thanks.
Hi,
Please do not disable the tests, especially the ones that fail saying
UNSAFE. It saysUNSAFEbecause it indicates that the code being tested is failing in an unsafe way, which could be a security vulnerability.
OK. I'll wait for a resolution before pushing the updated package to the Guix repository.
You listed the package versions, but what about the Emacs version?
Emacs 27.1
Also, to be clear, are you packaging this for Guix officially, or is this just for your personal use?
Officially. The package in Guix is currently at version 0.3.2, and cannot be used because its test suite is also failing. I've updated it to 0.5 and noticed the 3 test failures above.
Thanks!
In case you already have Guix or have an interest in trying it, the following two patches can be applied to the master branch, and the failures should be reproducible via ./pre-inst-env guix build emacs-org-ql. from the built Guix source tree.
0001-gnu-Add-emacs-with-simulated-input.patch.gz 0002-gnu-emacs-org-ql-Update-to-0.5.patch.gz
OK. I'll wait for a resolution before pushing the updated package to the Guix repository.
You can see here that all of the tests are passing on CI, including on Emacs 26.3, 27.1, and 28.0: https://github.com/alphapapa/org-ql/runs/1439654398 If they aren't passing on Guix, I wouldn't know why or how to diagnose it.
Officially. The package in Guix is currently at version 0.3.2, and cannot be used because its test suite is also failing. I've updated it to 0.5 and noticed the 3 test failures above.
FYI, I made 9 stable releases in the 0.4 series before releasing 0.5.
In case you already have Guix or have an interest in trying it, the following two patches can be applied to the master branch
Why are you applying patches before building? If there are bugs in 0.5, please report them so I can fix them and release a 0.5.1.
Upon closer inspection, I see that the two UNSAFE failures are due to the fact that you are using [email protected]. org-ql 0.5 explicitly depends on org-super-agenda 1.2, because that version has the necessary support to operate these features safely. That's why I released org-super-agenda 1.2 before releasing org-ql 0.5.
https://github.com/alphapapa/org-ql/blob/3924b023e0f1e8fa93cfa79bdd71f9f0c0fb4679/org-ql.el#L6
If there are any guidelines for packaging Emacs packages for Guix, I guess they should be updated to emphasize that the versions in the Package-Requires header are meaningful (at least, when the package authors use them properly, which I try to do). Emacs's built-in package library respects them, and it would not allow a user to install org-ql 0.5 without having org-super-agenda 1.2 installed first.
As for the third test failure, I don't know what would cause that, but I'm guessing it's due to breaking changes in Org 9.4. I haven't had time to upgrade to Org 9.4 and test with it yet. OTOH, it might be something else causing that failure, and it would be hard for me to diagnose it without being able to reproduce it. However, that test failure could be safely ignored; at worst, it would be a very minor bug which wouldn't impair 99% of the package's functionality.
Not that I'm complaining about anyone's volunteer work, of course, but it's a little disappointing that the version of org-ql in Guix is over a year old, and that there've been 10 stable releases since then. So I appreciate your work to update this package in Guix.
Since the third test failure could indicate a bug on Org 9.4, I'll use this issue to track it.
Hi!
Upon closer inspection, I see that the two
UNSAFEfailures are due to the fact that you are using[email protected].org-ql0.5 explicitly depends onorg-super-agenda1.2, because that version has the necessary support to operate these features safely. That's why I releasedorg-super-agenda1.2 before releasingorg-ql0.5.
OK, sorry for not having checked the Package-Requires header. Updating org-super-agenda to version 1.2 indeed resolves these two failures.
https://github.com/alphapapa/org-ql/blob/3924b023e0f1e8fa93cfa79bdd71f9f0c0fb4679/org-ql.el#L6
If there are any guidelines for packaging Emacs packages for Guix, I guess they should be updated to emphasize that the versions in the
Package-Requiresheader are meaningful (at least, when the package authors use them properly, which I try to do). Emacs's built-inpackagelibrary respects them, and it would not allow a user to installorg-ql0.5 without havingorg-super-agenda1.2 installed first.
Interesting, I wasn't aware of that. There were no guidelines for Emacs packaging in Guix, but I've just addressed this (including your suggestion) with this commit: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=15fba3b13d543bbc1f13907e659c354a5b6f28ec.
As for the third test failure, I don't know what would cause that, but I'm guessing it's due to breaking changes in Org 9.4. I haven't had time to upgrade to Org 9.4 and test with it yet. OTOH, it might be something else causing that failure, and it would be hard for me to diagnose it without being able to reproduce it. However, that test failure could be safely ignored; at worst, it would be a very minor bug which wouldn't impair 99% of the package's functionality.
Good to know! I pushed the update to the Guix emacs-org-ql package here: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=ca11bbc8f35f6d367cce7cfd6d38d7e979c3ef49.
Not that I'm complaining about anyone's volunteer work, of course, but it's a little disappointing that the version of
org-qlin Guix is over a year old, and that there've been 10 stable releases since then. So I appreciate your work to update this package in Guix.
Eh, thanks! I saw on Reddit you were dabbling with Guix; hopefully you'll like it enough to want to get involved in keeping those packages fresh :-).
Thanks again for the prompt feedback.
OK, sorry for not having checked the
Package-Requiresheader. Updatingorg-super-agendato version 1.2 indeed resolves these two failures.
Thanks.
Interesting, I wasn't aware of that. There were no guidelines for Emacs packaging in Guix, but I've just addressed this (including your suggestion) with this commit: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=15fba3b13d543bbc1f13907e659c354a5b6f28ec.
Very well-written! Thanks.
As for the third test failure, I don't know what would cause that, but I'm guessing it's due to breaking changes in Org 9.4. I haven't had time to upgrade to Org 9.4 and test with it yet. OTOH, it might be something else causing that failure, and it would be hard for me to diagnose it without being able to reproduce it. However, that test failure could be safely ignored; at worst, it would be a very minor bug which wouldn't impair 99% of the package's functionality.
Good to know! I pushed the update to the Guix
emacs-org-qlpackage here: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=ca11bbc8f35f6d367cce7cfd6d38d7e979c3ef49.
Thanks. By the way, although that buttercup command may work, it's not how I test the package, so you might prefer to run make test. That way, if I change something in the future, that command should continue to work.
Not that I'm complaining about anyone's volunteer work, of course, but it's a little disappointing that the version of
org-qlin Guix is over a year old, and that there've been 10 stable releases since then. So I appreciate your work to update this package in Guix.Eh, thanks! I saw on Reddit you were dabbling with Guix; hopefully you'll like it enough to want to get involved in keeping those packages fresh :-).
Yeah, I'm interested in Guix, but it seems like a lot to wrap my head around. And, to be honest, I'm not sure if it's the most appropriate way to manage Emacs packages, since there are already powerful tools in Emacs for that (e.g. Quelpa, Straight, Borg, etc). But I'm happy to have my packages in Guix, anyway. :)
Thanks. By the way, although that
buttercupcommand may work, it's not how I test the package, so you might prefer to runmake test. That way, if I change something in the future, that command should continue to work.
This seems to depend on the git repository metadata being available:
starting phase `check'
ERROR (2020-12-18 16:26:30): No files specified and not in a git repo.
make: *** [Makefile:53: test] Error 1
command "make" "test" "-j" "4" failed with status 2
The .git/ directory is deleted from the source checkout in Guix. Here's the comment explaining why: https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/git.scm?id=2b76179ecd951172288f5f6f78402d9304d2da41#n73
Eh, thanks! I saw on Reddit you were dabbling with Guix; hopefully you'll like it enough to want to get involved in keeping those packages fresh :-).
Yeah, I'm interested in Guix, but it seems like a lot to wrap my head around. And, to be honest, I'm not sure if it's the most appropriate way to manage Emacs packages, since there are already powerful tools in Emacs for that (e.g. Quelpa, Straight, Borg, etc). But I'm happy to have my packages in Guix, anyway. :)
What's neat about Guix is that it enables you to manage any software in a uniform and declarative way (you can forget about all these domain-specific package managers and their specific quirks). Also, I don't know about the tools you listed above, but in Guix your whole Emacs package collections is built against an exact Emacs version, which ensures proper operation (no byte compilation surprises when the Emacs version gets bumped, etc.). You can also have multiple profiles to manage different set of Emacs packages independently. Anyway, I'm obviously biased but I encourage you to overcome the initial "wow, that's a lot to take in" reaction. It pays off, in my opinion :-).
This seems to depend on the git repository metadata being available:
starting phase `check' ERROR (2020-12-18 16:26:30): No files specified and not in a git repo. make: *** [Makefile:53: test] Error 1 command "make" "test" "-j" "4" failed with status 2
Yes, makem.sh operates on files that are checked in to git. This prevents spurious Elisp files from affecting output, and allows the developer to, e.g. write a scratch.el file in the directory without having to manually exclude it from tests.
The .git/ directory is deleted from the source checkout in Guix. Here's the comment explaining why: https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/git.scm?id=2b76179ecd951172288f5f6f78402d9304d2da41#n73
I understand the principle of reproducible builds, but it would seem better to exclude or ignore the .git directory from the inputs rather than delete it. I doubt my trivial makem.sh script is the only build/test system that consults .git while operating. And, in general, it would seem better to build a package in an environment as close as possible to the one used by the developer; deleting .git is far from that.
in Guix your whole Emacs package collections is built against an exact Emacs version, which ensures proper operation (no byte compilation surprises when the Emacs version gets bumped, etc.).
Emacs byte-code is nearly universally backward-compatible, so that should very rarely, if ever, be an issue.
Thinking as a user rather than a developer, Emacs packages are upgraded on, e.g. MELPA and ELPA frequently. If I were to use Guix to install Emacs packages I use, it could mean waiting a very long time before I could get updates to them (e.g. how this package was stuck at 0.3.2 for over a year--again, thankful for your work, not a complaint, etc).
While I like Guix a lot, it's these kinds of small-yet-significant issues that would make it hard for me to recommend using it to manage Emacs packages.
Anyway, I'm obviously biased but I encourage you to overcome the initial "wow, that's a lot to take in" reaction. It pays off, in my opinion :-).
I'd love to use it as my system distro, theoretically. But IME, so far, just upgrading a few packages takes a very long time compared to on Debian (e.g. that Reddit thread, in which I only had a single package chosen for installation in the profile, vlc, yet it insisted on installing everything from MariaDB and Postgres to TeXLive). I feel like it isn't yet practical for me to use as my main system. I hope it gets there in a few years.
Hi again,
The .git/ directory is deleted from the source checkout in Guix. Here's the comment explaining why: https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/git.scm?id=2b76179ecd951172288f5f6f78402d9304d2da41#n73
I understand the principle of reproducible builds, but it would seem better to exclude or ignore the
.gitdirectory from the inputs rather than delete it. I doubt my trivialmakem.shscript is the only build/test system that consults.gitwhile operating. And, in general, it would seem better to build a package in an environment as close as possible to the one used by the developer; deleting.gitis far from that.
Deleting the .git directory allows the result of fetching the sources to be reproducible (if you considered .git when computing the hash of the git fetched sources, it would change every time there was some new change made to the repo -- we want it to be stable through time) for a given revision. Depending on git ls-files output is common in some circles (e.g., Ruby); in Guix this is typically worked around using find instead.
in Guix your whole Emacs package collections is built against an exact Emacs version, which ensures proper operation (no byte compilation surprises when the Emacs version gets bumped, etc.).
Emacs byte-code is nearly universally backward-compatible, so that should very rarely, if ever, be an issue.
I seem to recall the transition from 26 to 27 wasn't that smooth, byte compilation wise. Perhaps I'm wrong.
Thinking as a user rather than a developer, Emacs packages are upgraded on, e.g. MELPA and ELPA frequently. If I were to use Guix to install Emacs packages I use, it could mean waiting a very long time before I could get updates to them (e.g. how this package was stuck at 0.3.2 for over a year--again, thankful for your work, not a complaint, etc).
Some packages are very well maintained; but it really depends of the packages. In any case updating packages (especially Emacs ones) is usually a simple task, and it's easy to run a local Guix tree with modifications (./pre-inst-env guix install my-update-package) or setup a personal Guix channel.
While I like Guix a lot, it's these kinds of small-yet-significant issues that would make it hard for me to recommend using it to manage Emacs packages.
Anyway, I'm obviously biased but I encourage you to overcome the initial "wow, that's a lot to take in" reaction. It pays off, in my opinion :-).
I'd love to use it as my system distro, theoretically. But IME, so far, just upgrading a few packages takes a very long time compared to on Debian (e.g. that Reddit thread, in which I only had a single package chosen for installation in the profile,
vlc, yet it insisted on installing everything from MariaDB and Postgres to TeXLive). I feel like it isn't yet practical for me to use as my main system. I hope it gets there in a few years.
VLC does indeed depend on postgres and mariadb through qtbase. The same is true for qtbase in Debian (https://salsa.debian.org/qt-kde-team/qt/qtbase/-/blob/master/debian/control). It requires libpq-dev, which is postgres. It doesn't seem to be built with support for mariadb, though. Debian splits qtbase in many sub-packages, and the database support is made optional, so unless something explicitly depends on libqt5sql5-psql, libpq wouldn't need to be pulled in, I believe. In Guix we can split things in different outputs. This could be used to reduce the closure of the main output of qtbase.
Thanks for the feedback!
@Apteryks I worked on the tests recently and they should all pass now. Can you verify that, and if so, remove the disabled test from the package definition? Thanks for your work.
Hello! It seems it was already taken care of:
Date: Fri Mar 10 19:09:03 2023 +0100
gnu: emacs-org-ql: Update to 0.7.
* gnu/packages/emacs-xyz.scm (emacs-org-ql): Update to 0.7.
[arguments]<#:phases>: Remove unnecessary test fix.
Thanks for the heads-up!