quicklisp-client icon indicating copy to clipboard operation
quicklisp-client copied to clipboard

:DEFSYSTEM-DEPENDS-ON dependencies

Open luismbo opened this issue 10 years ago • 17 comments

I get the following error if I don't have CFFI installed beforehand when attempting to load trivial-features-tests whose ASDF system depends on cffi-grovel via :defsystem-depends-on:

* (ql:quickload "trivial-features-tests")
Error while trying to load definition for system
trivial-features-tests from pathname
/Users/luis/src/lisp/trivial-features/trivial-features-tests.asd:

   Component ASDF/USER::CFFI-GROVEL not found, required by NIL
   [Condition of type ASDF/FIND-SYSTEM:LOAD-SYSTEM-DEFINITION-ERROR]

Restarts:
 0: [RETRY] Retry EVAL of current toplevel form.
 1: [CONTINUE] Ignore error and continue loading file "/Users/luis/src/lisp/trivial-features/trivial-features-tests.asd".
 2: [ABORT] Abort loading file "/Users/luis/src/lisp/trivial-features/trivial-features-tests.asd".
 3: [RETRY] Retry ASDF operation.
 4: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the configuration.
 5: [ABORT] Give up on "trivial-features-tests"
 --more--

Backtrace:
  0: ((FLET #:HANDLER1584 :IN ASDF/FIND-SYSTEM:LOAD-ASD) Component ASDF/USER::CFFI-GROVEL not found, required by NIL)
  1: (SIGNAL Component ASDF/USER::CFFI-GROVEL not found, required by NIL)
  2: (ERROR ASDF/FIND-COMPONENT:MISSING-DEPENDENCY :REQUIRED-BY NIL :REQUIRES ASDF/USER::CFFI-GROVEL)
  3: (ASDF/FIND-COMPONENT:RESOLVE-DEPENDENCY-NAME NIL ASDF/USER::CFFI-GROVEL NIL)
  4: ((LAMBDA NIL :IN ASDF/PARSE-DEFSYSTEM:REGISTER-SYSTEM-DEFINITION))
  5: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ASDF/PARSE-DEFSYSTEM:DEFSYSTEM ASDF/USER::TRIVIAL-FEATURES-TESTS :DESCRIPTION "Unit tests for TRIVIAL-FEATURES." :DEFSYSTEM-DEPENDS-ON (ASDF/USER::CFFI-GROVEL) ...) #<NU..
  6: (EVAL-TLF (ASDF/PARSE-DEFSYSTEM:DEFSYSTEM ASDF/USER::TRIVIAL-FEATURES-TESTS :DESCRIPTION "Unit tests for TRIVIAL-FEATURES." :DEFSYSTEM-DEPENDS-ON (ASDF/USER::CFFI-GROVEL) ...) 1 #<NULL-LEXENV>)
  7: ((FLET SB-FASL::EVAL-FORM :IN SB-INT:LOAD-AS-SOURCE) (ASDF/PARSE-DEFSYSTEM:DEFSYSTEM ASDF/USER::TRIVIAL-FEATURES-TESTS :DESCRIPTION "Unit tests for TRIVIAL-FEATURES." :DEFSYSTEM-DEPENDS-ON (ASDF/USER:..
  8: (SB-INT:LOAD-AS-SOURCE #<SB-SYS:FD-STREAM for "file /Users/luis/src/lisp/trivial-features/trivial-features-tests.asd" {100580C783}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
  9: ((FLET SB-FASL::LOAD-STREAM :IN LOAD) #<SB-SYS:FD-STREAM for "file /Users/luis/src/lisp/trivial-features/trivial-features-tests.asd" {100580C783}> NIL)
 10: (LOAD #P"/Users/luis/src/lisp/trivial-features/trivial-features-tests.asd" :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST T :EXTERNAL-FORMAT :UTF-8)
 11: (UIOP/UTILITY:CALL-WITH-MUFFLED-CONDITIONS #<CLOSURE (LAMBDA NIL :IN UIOP/LISP-BUILD:LOAD*) {100580845B}> ("Overwriting already existing readtable ~S." #(#:FINALIZERS-OFF-WARNING :ASDF-FINALIZERS)))
 12: ((LAMBDA NIL :IN ASDF/FIND-SYSTEM:LOAD-ASD))
 13: (SB-IMPL::%WITH-STANDARD-IO-SYNTAX #<CLOSURE (LAMBDA NIL :IN ASDF/FIND-SYSTEM:LOAD-ASD) {10058082FB}>)
 14: ((LAMBDA NIL :IN ASDF/SYSTEM:FIND-SYSTEM))
 15: (ASDF/CACHE:CONSULT-ASDF-CACHE (ASDF/SYSTEM:FIND-SYSTEM "trivial-features-tests") #<CLOSURE (LAMBDA NIL :IN ASDF/SYSTEM:FIND-SYSTEM) {10057F387B}>)
 16: (ASDF/CACHE:CALL-WITH-ASDF-CACHE #<CLOSURE (LAMBDA NIL :IN ASDF/SYSTEM:FIND-SYSTEM) {10057F387B}> :OVERRIDE NIL :KEY (ASDF/SYSTEM:FIND-SYSTEM "trivial-features-tests"))
 17: ((:METHOD ASDF/SYSTEM:FIND-SYSTEM (STRING)) "trivial-features-tests" NIL) [fast-method]
 18: ((LABELS QUICKLISP-CLIENT::RECURSE :IN QUICKLISP-CLIENT::COMPUTE-LOAD-STRATEGY) "trivial-features-tests")
 19: ((LAMBDA NIL :IN QUICKLISP-CLIENT::COMPUTE-LOAD-STRATEGY)) [external]
 20: (QL-DIST::CALL-WITH-CONSISTENT-DISTS #<CLOSURE (LAMBDA NIL :IN QUICKLISP-CLIENT::COMPUTE-LOAD-STRATEGY) {10057F37DB}>)
 21: (QUICKLISP-CLIENT::COMPUTE-LOAD-STRATEGY #<unavailable argument>)
 22: (QUICKLISP-CLIENT::AUTOLOAD-SYSTEM-AND-DEPENDENCIES "trivial-features-tests" :PROMPT NIL)
 23: ((:METHOD QL-IMPL-UTIL::%CALL-WITH-QUIET-COMPILATION (T T)) #<unavailable argument> #<CLOSURE (FLET QUICKLISP-CLIENT::QL :IN QUICKLISP-CLIENT:QUICKLOAD) {10057F073B}>) [fast-method]
 24: ((:METHOD QL-IMPL-UTIL::%CALL-WITH-QUIET-COMPILATION :AROUND (QL-IMPL:SBCL T)) #<QL-IMPL:SBCL {10038AE3F3}> #<CLOSURE (FLET QUICKLISP-CLIENT::QL :IN QUICKLISP-CLIENT:QUICKLOAD) {10057F073B}>) [fast-me..
 25: ((:METHOD QUICKLISP-CLIENT:QUICKLOAD (T)) #<unavailable argument> :PROMPT NIL :VERBOSE NIL) [fast-method]
 26: (QL-DIST::CALL-WITH-CONSISTENT-DISTS #<CLOSURE (LAMBDA NIL :IN QUICKLISP-CLIENT:QUICKLOAD) {10057E541B}>)
 27: (SB-INT:SIMPLE-EVAL-IN-LEXENV (QUICKLISP-CLIENT:QUICKLOAD "trivial-features-tests") #<NULL-LEXENV>)
 28: (EVAL (QUICKLISP-CLIENT:QUICKLOAD "trivial-features-tests"))

I'm using quicklisp-client 2014-12-10 and ASDF 3.1.3.

luismbo avatar Dec 18 '14 00:12 luismbo

I chalked this up to not wrapping the load planning in the auto-load handler, but it's trickier than that. :DEFSYSTEM-DEPENDS-ON signals one kind of error on a missing component, and :DEPENDS-ON signals a completely different (and much more useful) error. In fact, the former error hides the "real" error in an un-exported slot on the condition.

So, I realize this is annoying, and I want to fix it, but it's a little trickier than I thought it might be at first glance, so it may take a while, and some unpleasant fudging with ASDF internals.

quicklisp avatar Dec 18 '14 01:12 quicklisp

No problem, I work around this issue by loading cffi-grovel explicitly beforehand. Thanks for looking into it.

luismbo avatar Dec 18 '14 01:12 luismbo

Ok, with the January 2015 release, the dependencies are more explicit in the indexes, and I believe this issue is fixed. If you have any trouble with the new release, please let me know.

https://github.com/quicklisp/quicklisp-controller/commit/8d7a8b5f664f6b619047bb1adaa1cf421a9138d3 is the commit that cleaned it up.

quicklisp avatar Jan 13 '15 19:01 quicklisp

I'm still getting an error. https://travis-ci.org/trivial-features/trivial-features/jobs/47704343#L834 Am I doing something wrong?

luismbo avatar Jan 20 '15 21:01 luismbo

Ok, I see why that's happening, and why I thought it was fixed but it isn't (for this scenario).

The part I fixed is when loading trivial-features-tests (or certain iolib things) from Quicklisp. But the problem with the :defsystem-depends-on dependencies remains when the system is being overridden via ASDF. I'll keep digging for that solution.

quicklisp avatar Jan 20 '15 22:01 quicklisp

This also happens in the following case: A local-projects system defsystem-depends-on another local-projects system. The dependencies of the second system are not fetched through quicklisp when the first system is quickloaded. I suppose this falls into the same "ASDF overrides" category as mentioned.

Shinmera avatar Jun 10 '15 16:06 Shinmera

There's another issue at play. The error signaled when a :depends-on system is absent is different from the error signaled when a :defsystem-depends-on system is absent. So my hoped-for fix (wrapping the handler over a bigger scope) doesn't work.

quicklisp avatar Jun 10 '15 16:06 quicklisp

I've just run into similar issue. Isolated test case: http://paste.lisp.org/display/160873#2

otwieracz avatar Nov 26 '15 13:11 otwieracz

You might already be aware of this, but loading a system that uses :defsystem-depends-on directly from quicklisp still yields an error: with the 2016-08-25 quicklisp dist and the 2016-02-22 client, (ql:quickload :quri-test) returns an error about :prove-asdf being a missing dependency, even though :prove-asdf can be quickload-ed directly.

mtstickney avatar Sep 11 '16 20:09 mtstickney

I have this problem when trying to quickload my project:


Error while trying to load definition for system clox from
pathname /home/malice/Programming/Lisp/clox/clox.asd:
   Component :DEPLOY not found, required by NIL

Not only that, repeating the command yields no error, but the system is not loaded.

Quickloading :deploy (the dependency) fixes the issue.

The project, as you can see, is in the path supplied to asdf by :tree option in projects.conf file.

My versions are:

CL-USER> (ql:client-version)
"2016-02-22"
CL-USER> (ql:update-all-dists)
1 dist to check.
You already have the latest version of "quicklisp": 2017-07-25.
NIL

The relevant path of asd file:

:defsystem-depends-on (:deploy)

After quickloading the system deploy manually, the error is fixed.

MatthewRock avatar Aug 25 '17 22:08 MatthewRock

I'm having the same problem with :defsystem-depends-on (#:prove) as prove's readme suggest to use.

Is there something somebody could do about this?

What's the error thrown by defsystem-depends-on and where would one look in quicklisp to (try to) fix that?

fstamour avatar Jan 18 '18 02:01 fstamour

I noticed that asdf 3.3.0 changed to throw a missing-dependency error when missing a defsystem dependency. So, I combine that with the idea behind #122 of moving (compute-load-strategy name) inside the handler-case. Additionally, I had to remove the check whether the condition's parent is a system, since it's nil when a defsystem dependency can't be found and I couldn't find any spot in asdf's source that wouldn't indicate a system to try to load. The fix won't work if asdf is older than 3.3.0, but it won't break anything.

I can make this a pull request, but I wanted to get feedback on this first given that there are already 2 unmerged pull requests trying to fix this (#122 and #128).

(defun autoload-system-and-dependencies (name &key prompt)
  "Try to load the system named by NAME, automatically loading any
Quicklisp-provided systems first, and catching ASDF missing
dependencies too if possible."
  (setf name (string-downcase name))
  (with-simple-restart (abort "Give up on ~S" name)
    (let ((tried-so-far (make-hash-table :test 'equalp)))
      (tagbody
       retry
       (handler-case
         (let ((strategy (compute-load-strategy name)))
           (show-load-strategy strategy)
           (when (or (not prompt)
                     (press-enter-to-continue))
             (apply-load-strategy strategy)))
         (asdf:missing-dependency-of-version (c)
           ;; Nothing Quicklisp can do to recover from this, so just
           ;; resignal
           (error c))
         (asdf:missing-dependency (c)
           (let ((missing (asdf::missing-requires c)))
             (if (gethash missing tried-so-far)
                (error "Dependency looping -- already tried to load ~
                            ~A" missing)
                (setf (gethash missing tried-so-far) missing))
             (autoload-system-and-dependencies missing
                                              :prompt prompt)
             (go retry))))))
    name))

neil-lindquist avatar Jul 19 '18 22:07 neil-lindquist

This is a real problem that I'd like to fix but it just hasn't come to the top of my list of things to examine and think hard about, sorry.

On Thu, Jul 19, 2018 at 6:51 PM, Neil Lindquist [email protected] wrote:

I noticed that asdf 3.3.0 changed to throw a missing-dependency error when missing a defsystem dependency. So, I combine that with the idea behind #122 https://github.com/quicklisp/quicklisp-client/pull/122 of moving (compute-load-strategy name) inside the handler-case. Additionally, I had to remove the check whether the condition's parent is a system, since it's nil when a defsystem dependency can't be found and I couldn't find any spot in asdf's source that wouldn't indicate a system to try to load. The fix won't work if asdf is older than 3.3.0, but it won't break anything.

I can make this a pull request, but I wanted to get feedback on this first given that there are already 2 unmerged pull requests trying to fix this ( #122 https://github.com/quicklisp/quicklisp-client/pull/122 and #128 https://github.com/quicklisp/quicklisp-client/pull/128).

(defun autoload-system-and-dependencies (name &key prompt) "Try to load the system named by NAME, automatically loading any Quicklisp-provided systems first, and catching ASDF missing dependencies too if possible." (setf name (string-downcase name)) (with-simple-restart (abort "Give up on ~S" name) (let ((tried-so-far (make-hash-table :test 'equalp))) (tagbody retry (handler-case (let ((strategy (compute-load-strategy name))) (show-load-strategy strategy) (when (or (not prompt) (press-enter-to-continue)) (apply-load-strategy strategy))) (asdf:missing-dependency-of-version (c) ;; Nothing Quicklisp can do to recover from this, so just ;; resignal (error c)) (asdf:missing-dependency (c) (let ((missing (asdf::missing-requires c))) (if (gethash missing tried-so-far) (error "Dependency looping -- already tried to load ~ ~A" missing) (setf (gethash missing tried-so-far) missing)) (autoload-system-and-dependencies missing :prompt prompt) (go retry)))))) name))

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/quicklisp/quicklisp-client/issues/108#issuecomment-406438137, or mute the thread https://github.com/notifications/unsubscribe-auth/AAT2rafcXAnBTozqBU3lA-Jc0sEWPnYVks5uIQ10gaJpZM4DJzYe .

quicklisp avatar Aug 10 '18 00:08 quicklisp

Any update on this? It's still biting me with cl-bio and the current SBCL/ASDF combo (that's another story...).

slyrus avatar Dec 24 '19 07:12 slyrus

@xach has done recent work since #sbcl20 in https://github.com/quicklisp/ddo-bug. I think he has a patch to update the client. I think we're trying to figure out if ASDF3 should be patched at the same time, but I haven't had time to do the legwork to figure out if we should change the signal reporting behavior.

easye avatar Dec 27 '19 14:12 easye

This should be fixed with the latest client. Please try it and let me know if it helps.

xach avatar Jan 04 '20 23:01 xach

FTR, i have just saw a commented out KLUDGE in the build.sh script, and the script still works... so, i guess it's been fixed.

attila-lendvai avatar Dec 01 '21 14:12 attila-lendvai