qlot update fails regularily

[justjoheinz]

Describe the bug

> qlot update                                                                  
Reading 'xxx/Projects/lisp/rockets/qlfile'...
⨯ [1/4] quicklisp  Failed to install
● [2/4] cl-liballegro  Retrieving the git ref from GitHub.
● [4/4] ultralisp  Fetching the distinfo.
● [3/4] livesupport  Retrieving the git ref from GitHub.

Unexpected error: The value
                    NIL
                  is not of type
                    (OR STRING PATHNAME SYNONYM-STREAM FILE-STREAM)
                  when binding SB-KERNEL:FILENAME
This can be a bug of Qlot.

when doing qlot upgrade. Repeating the command will eventually suceed.

Reproducible steps

Error messages

Stacktrace as per request

Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {7005BB4023}>
0: ((LAMBDA NIL :IN UIOP/IMAGE:PRINT-BACKTRACE))
1: ((FLET "THUNK" :IN UIOP/STREAM:CALL-WITH-SAFE-IO-SYNTAX))
2: (SB-IMPL::%WITH-STANDARD-IO-SYNTAX #<FUNCTION (FLET "THUNK" :IN UIOP/STREAM:CALL-WITH-SAFE-IO-SYNTAX) {1054B262B}>)
3: (UIOP/STREAM:CALL-WITH-SAFE-IO-SYNTAX #<FUNCTION (LAMBDA NIL :IN UIOP/IMAGE:PRINT-BACKTRACE) {70097B56EB}> :PACKAGE :CL)
4: (UIOP/IMAGE:PRINT-CONDITION-BACKTRACE #<TYPE-ERROR expected-type: (OR STRING PATHNAME SYNONYM-STREAM FILE-STREAM) datum: NIL> :STREAM #<SB-SYS:FD-STREAM for "file /var/folders/vr/t70kp3dd5m14_99ngdw8gdv40000gn/T/qlot-error-5GEXGEG9.log" {70097B5553}> :COUNT NIL)
5: ((FLET "BEFORE29" :IN QLOT/CLI:QLOT-COMMAND) #<SB-SYS:FD-STREAM for "file /var/folders/vr/t70kp3dd5m14_99ngdw8gdv40000gn/T/qlot-error-5GEXGEG9.log" {70097B5553}> #P"/var/folders/vr/t70kp3dd5m14_99ngdw8gdv40000gn/T/qlot-error-5GEXGEG9.log")
6: (UIOP/STREAM:CALL-WITH-TEMPORARY-FILE #<FUNCTION (FLET "BEFORE29" :IN QLOT/CLI:QLOT-COMMAND) {1054B211B}> :WANT-STREAM-P T :WANT-PATHNAME-P T :DIRECTION :OUTPUT :KEEP T :AFTER NIL :DIRECTORY NIL :TYPE "log" :PREFIX "qlot-error-" :SUFFIX "" :ELEMENT-TYPE NIL :EXTERNAL-FORMAT NIL)
7: ((LAMBDA (QLOT/CLI::C) :IN QLOT/CLI:QLOT-COMMAND) #<TYPE-ERROR expected-type: (OR STRING PATHNAME SYNONYM-STREAM FILE-STREAM) datum: NIL>)
8: (SB-KERNEL::%SIGNAL #<TYPE-ERROR expected-type: (OR STRING PATHNAME SYNONYM-STREAM FILE-STREAM) datum: NIL>)
9: (ERROR #<TYPE-ERROR expected-type: (OR STRING PATHNAME SYNONYM-STREAM FILE-STREAM) datum: NIL>)
10: (LPARALLEL.KERNEL-UTIL::RECEIVE-RESULTS #S(LPARALLEL.KERNEL:CHANNEL :QUEUE #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX "Anonymous lock" free owner=0> :CVAR #<SB-THREAD:WAITQUEUE Anonymous condition variable {700973DB13}>) :KERNEL #<LPARALLEL.KERNEL:KERNEL :NAME "lparallel" :WORKER-COUNT 4 :USE-CALLER NIL :ALIVE T :SPIN-COUNT 2000 {7009E10023}>) #<unavailable argument> #<unavailable argument>)
11: (LPARALLEL.COGNATE::PMAP-ITERATE #<FUNCTION MAPC> #<FUNCTION (LAMBDA (QLOT/PROGRESS::JOB) :IN QLOT/PROGRESS:RUN-IN-PARALLEL) {7009E155BB}> ((#<QLOT/SOURCE/DIST:SOURCE-DIST https://beta.quicklisp.org/dist/quicklisp.txt LATEST {7009E12E23}> #<QLOT/SOURCE/GITHUB:SOURCE-GITHUB resttime/cl-liballegro #1=<undetermined> {7009E10633}> #<QLOT/SOURCE/DIST:SOURCE-DIST https://dist.ultralisp.org/ LATEST {7009E101A3}> #<QLOT/SOURCE/GITHUB:SOURCE-GITHUB cbaggers/livesupport #1# {7009E10803}>)) 4 4)
12: (LPARALLEL.COGNATE:PMAPC #<FUNCTION (LAMBDA (QLOT/PROGRESS::JOB) :IN QLOT/PROGRESS:RUN-IN-PARALLEL) {7009E155BB}> (#<QLOT/SOURCE/DIST:SOURCE-DIST https://beta.quicklisp.org/dist/quicklisp.txt LATEST {7009E12E23}> #<QLOT/SOURCE/GITHUB:SOURCE-GITHUB resttime/cl-liballegro #1=<undetermined> {7009E10633}> #<QLOT/SOURCE/DIST:SOURCE-DIST https://dist.ultralisp.org/ LATEST {7009E101A3}> #<QLOT/SOURCE/GITHUB:SOURCE-GITHUB cbaggers/livesupport #1# {7009E10803}>))
13: (QLOT/PROGRESS:RUN-IN-PARALLEL #<FUNCTION (LAMBDA (QLOT/INSTALL::SOURCE) :IN QLOT/INSTALL::APPLY-QLFILE-TO-QLHOME) {7009E101CB}> (#<QLOT/SOURCE/DIST:SOURCE-DIST https://beta.quicklisp.org/dist/quicklisp.txt LATEST {7009E12E23}> #<QLOT/SOURCE/GITHUB:SOURCE-GITHUB resttime/cl-liballegro #1=<undetermined> {7009E10633}> #<QLOT/SOURCE/DIST:SOURCE-DIST https://dist.ultralisp.org/ LATEST {7009E101A3}> #<QLOT/SOURCE/GITHUB:SOURCE-GITHUB cbaggers/livesupport #1# {7009E10803}>) :CONCURRENCY 4 :JOB-HEADER-FN #<FUNCTION (LAMBDA (QLOT/INSTALL::SOURCE) :IN QLOT/INSTALL::APPLY-QLFILE-TO-QLHOME) {7009E1558B}> :FAILED-FN #<FUNCTION (LAMBDA NIL :IN QLOT/INSTALL::APPLY-QLFILE-TO-QLHOME) {70092E1EFB}>)
14: (QLOT/INSTALL::APPLY-QLFILE-TO-QLHOME #P"/Users/markusklink/Projects/lisp/rockets/qlfile" #P"/Users/markusklink/Projects/lisp/rockets/.qlot/" :IGNORE-LOCK T :PROJECTS NIL :CACHE-DIRECTORY NIL :CONCURRENCY NIL)
15: (QLOT/INSTALL:UPDATE-QLFILE #P"/Users/markusklink/Projects/lisp/rockets/qlfile" :QUICKLISP-HOME NIL :PROJECTS NIL :INSTALL-DEPS T :CACHE-DIRECTORY NIL :CONCURRENCY NIL)
16: (UIOP/FILESYSTEM:CALL-WITH-CURRENT-DIRECTORY #P"/Users/markusklink/Projects/lisp/rockets/" #<FUNCTION (LAMBDA NIL :IN QLOT/CLI::%QLOT-COMMAND) {7005BC078B}>)
17: (QLOT/CLI:QLOT-COMMAND "update")
18: (SB-INT:SIMPLE-EVAL-IN-LEXENV (APPLY (QUOTE ROS/SCRIPT/QLOT::MAIN) ROSWELL:*ARGV*) #<NULL-LEXENV>)
19: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ROSWELL:QUIT (APPLY (QUOTE ROS/SCRIPT/QLOT::MAIN) ROSWELL:*ARGV*)) #<NULL-LEXENV>)
20: (SB-EXT:EVAL-TLF (ROSWELL:QUIT (APPLY (QUOTE ROS/SCRIPT/QLOT::MAIN) ROSWELL:*ARGV*)) NIL NIL)
21: ((LABELS SB-FASL::EVAL-FORM :IN SB-INT:LOAD-AS-SOURCE) (ROSWELL:QUIT (APPLY (QUOTE ROS/SCRIPT/QLOT::MAIN) ROSWELL:*ARGV*)) NIL)
22: (SB-INT:LOAD-AS-SOURCE #<CONCATENATED-STREAM :STREAMS NIL {7005BC0313}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
23: ((LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) #<CONCATENATED-STREAM :STREAMS NIL {7005BC0313}> NIL)
24: (SB-FASL::CALL-WITH-LOAD-BINDINGS #<FUNCTION (LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) {1054B0B4B}> #<CONCATENATED-STREAM :STREAMS NIL {7005BC0313}> NIL #<CONCATENATED-STREAM :STREAMS NIL {7005BC0313}>)
25: (LOAD #<CONCATENATED-STREAM :STREAMS NIL {7005BC0313}> :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST :ERROR :EXTERNAL-FORMAT :DEFAULT)
26: ((FLET ROSWELL::BODY :IN ROSWELL:SCRIPT) #<SB-SYS:FD-STREAM for "file /Users/markusklink/.roswell/bin/qlot" {7005BC0083}>)
27: (ROSWELL:SCRIPT "/Users/markusklink/.roswell/bin/qlot" "update")
28: (ROSWELL:RUN ((:EVAL "(ros:quicklisp)") (:SCRIPT "/Users/markusklink/.roswell/bin/qlot" "update") (:QUIT NIL)))
29: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ROSWELL:RUN (QUOTE ((:EVAL "(ros:quicklisp)") (:SCRIPT "/Users/markusklink/.roswell/bin/qlot" "update") (:QUIT NIL)))) #<NULL-LEXENV>)
30: (EVAL (ROSWELL:RUN (QUOTE ((:EVAL "(ros:quicklisp)") (:SCRIPT "/Users/markusklink/.roswell/bin/qlot" "update") (:QUIT NIL)))))
31: (SB-IMPL::PROCESS-EVAL/LOAD-OPTIONS ((:EVAL . "(progn #-ros.init(cl:load \"/opt/homebrew/Cellar/roswell/23.10.14.114/etc/roswell/init.lisp\"))") (:EVAL . "(ros:run '((:eval\"(ros:quicklisp)\")(:script \"/Users/markusklink/.roswell/bin/qlot\"\"update\")(:quit ())))")))
32: (SB-IMPL::TOPLEVEL-INIT)
33: ((FLET SB-UNIX::BODY :IN SB-IMPL::START-LISP))
34: ((FLET "WITHOUT-INTERRUPTS-BODY-3" :IN SB-IMPL::START-LISP))
35: (SB-IMPL::%START-LISP)
Above backtrace due to this condition:
The value NIL is not of type (OR STRING PATHNAME SYNONYM-STREAM FILE-STREAM) when binding SB-KERNEL:FILENAME

Versions:

• Qlot version: 1.5.0
• Where did you get it from: ros update qlot
• OS: Mac OS
• Lisp: SBCL 2.4.1

[fukamachi]

It sounds like it's because your network is unstable. It might be solved by specifying --jobs 1 to turn off concurrent installation.

[justjoheinz]

I tried for some time now to use qlot upate --jobs 1 but it didn't make much of a difference. It is strange though, that repeating the command will succeed most of the times. Maybe the error could be handled better and a retry policy established within qlot?

[fosskers]

This occurs for me every time I do qlot update. Note it only fails when trying to update the ultralisp dist as defined like:

dist http://dist.ultralisp.org/

Also note that if I run it a second time, it succeeds. Some middle stage of the update seems to fail, but it seems to be downloading enough of what it needs for the second attempt to succeed.

[justjoheinz]

What @fosskers says :-) It seems to be always ultralisp and the second attempt succeeds. I am now on 1.5.8 and the problem still exists. Using the --jobs flag does not make a difference.

[justjoheinz]

1.5.9 can still reproduce this almost always. Is there anything I can provide to help debug the problem? Is there anything special with the ultralisp distribution? I guess it is the biggest?

[svetlyak40wt]

I notice update with —no-deps flag does not fail.

[fukamachi]

I still can't reproduce it. Can anyone share qlfile & qlfile.lock?

[fosskers]

qlfile

dist http://dist.ultralisp.org

lock file

("quicklisp" .
 (:class qlot/source/dist:source-dist
  :initargs (:distribution "https://beta.quicklisp.org/dist/quicklisp.txt" :%version :latest)
  :version "2023-10-21"))
("ultralisp" .
 (:class qlot/source/dist:source-dist
  :initargs (:distribution "https://dist.ultralisp.org" :%version :latest)
  :version "20240821211000"))

[fukamachi]

Thank you!! Now, I got the reason, and it will be fixed soon.

[justjoheinz]

THANK YOU! Just tried it 3 times in a row and works each time!