sly
sly copied to clipboard
joaotavora
Setting character element type to allow Unicode characters in symbols.
Cc @phoe or anyone with lispworks access.
Why at compile time? Why not only load time?
Doesn‘t need to be at compile time. Just thiught this might be a good place as I have seen no other „init“ hook in lispworks.lisp.
Compile-time is probably overkill - Slynk compiles fine without Unicode, it needs Unicode only at runtime.
(eval-when (:compile-toplevel :load-toplevel :execute)
(require "comm"))
;;; The default LW character element type is BASE-CHAR.
;;; However, we want Unicode in order to handle non-base-chars
;;; that can be sent from Sly into Slynk.
;;; See e.g. https://github.com/joaotavora/sly/discussions/513
(lw:set-default-character-element-type 'character)
I was actually thinking about adding the discussions link, but thought the source code will probably live longer than the link, or the discussions facility in GitHub. But OK, no prob I can add it.
What means overkill? Will it result in bigger fasls?
By "overkill" I meant that sly does not require being compiled with the default character type set to character whatsoever.
OK, I'm back at my PC now and want to thank you for figuring out the culprit.
I think that definitely setting the default element type for all streams would work, but I wonder if it's the best idea. That's because I suppose LispWorks users are counting on this non-standard efficiency gain of having base-char streams by default. If we set it globally just because they are using SLY, we negate those presumed gains, which could be annoying. This is just conjecture.
So, I was reading the SLIME issue that @phoe linked to https://github.com/slime/slime/issues/691 and see that the proposal there is to set the element type in some subset of streams essential for communication. It's a bit more sprinkly, but it should be considered I think (depending on whether or now you agree with the above conjecture, or the degree of agreement with said conjecture).
I'll try to come up with this alternative patch, so we can compare. Also, it's curious that SLIME hasn't fixed this yet in about 6 months (might just be SLIME maintainers are even lazier than me).
Let's start with this (100% untested) idea:
diff --git a/slynk/slynk-rpc.lisp b/slynk/slynk-rpc.lisp
index b4ab93f6..3f4d6fac 100644
--- a/slynk/slynk-rpc.lisp
+++ b/slynk/slynk-rpc.lisp
@@ -108,6 +108,8 @@ to NIL in her ~/.swankrc. Generally best left alone.")
(let ((c (read-char)))
(case c
(#\" (with-output-to-string (*standard-output*)
+ #+lispworks
+ (setf (stream-element-type *standard-output*) 'lw:simple-char)
(loop for c = (read-char) do
(case c
(#\" (return))
@mdbergmann can you test this patch and tell if this fixes the issue?
Won't modifying the arguments to w-o-t-s below work better here?
(with-output-to-string (*standard-output* nil
;; Explicitly setting output type for LispWorks
;; as it diverges from the CL specification here.
;; See http://www.lispworks.com/documentation/lw60/LW/html/lw-634.htm
#+lispworks :element-type #+lispworks 'character)
...)
If we do not do global patching, we'll probably need to patch all w-o-t-s and open and with-open-file forms as a start and then check if we missed anything.
So it seems (stream-element-type *standard-output*) is not setf'able? Getting an error doing this: Error: Undefined operator (SETF STREAM-ELEMENT-TYPE) in form ((SETF STREAM-ELEMENT-TYPE) #:|Store-Var-1363| #:FORM1364).
@phoe s variant (tried with both lw:simple-char and 'character`) works in that Slynk doesn't crash anymore, but now a type error is raised in Sly, assumingly when trying to print this character.
Backtrace:
0: [error printing frame]
1: [error printing frame]
2: [error printing frame]
3: ((SUBFUNCTION 1 (METHOD STREAM:STREAM-WRITE-STRING (SLYNK-GRAY::SLY-OUTPUT-STREAM T))))
4: ((SUBFUNCTION (FLET SLYNK-BACKEND:CALL-WITH-LOCK-HELD) (TOP-LEVEL-FORM 126)) ..)
5: [error printing frame]
6: [error printing frame]
7: [error printing frame]
8: [error printing frame]
9: [error printing frame]
10: [error printing frame]
11: [error printing frame]
12: (SYSTEM::%INVOKE :INVISIBLEP T)
13: [error printing frame]
14: [error printing frame]
15: ((SUBFUNCTION 1 (SUBFUNCTION 1 SLYNK-MREPL::MREPL-EVAL-1)))
16: (SLYNK::CALL-WITH-RETRY-RESTART "Retry SLY mREPL evaluation request." #<Closure 1 subfunction of (SUBFUNCTION 1 SLYNK-MREPL::MREPL-EVAL-1) 8020018F89>)
17: ((SUBFUNCTION 1 SLYNK-MREPL::MREPL-EVAL-1))
18: ((SUBFUNCTION 1 SLYNK::CALL-WITH-LISTENER))
19: (SLYNK::CALL-WITH-BINDINGS ((*PACKAGE* . #<The COMMON-LISP-USER package, 34/64 internal, 0/4 external>) (*DEFAULT-PATHNAME-DEFAULTS* . #P"") (*) (**) (***) (/) ...) ..)
20: [error printing frame]
21: [error printing frame]
22: (SLYNK:PROCESS-REQUESTS NIL)
23: ((SUBFUNCTION 1 (SUBFUNCTION 1 (SUBFUNCTION 1 SLYNK::SPAWN-CHANNEL-THREAD))))
24: ((SUBFUNCTION 1 (SUBFUNCTION 1 SLYNK::SPAWN-CHANNEL-THREAD)))
25: ((SUBFUNCTION (FLET SLYNK-BACKEND:CALL-WITH-DEBUGGER-HOOK) (TOP-LEVEL-FORM 50)) ..)
26: ((SUBFUNCTION 1 SLYNK::CALL-WITH-LISTENER))
27: (SLYNK::CALL-WITH-BINDINGS ((*PACKAGE* . #<The COMMON-LISP-USER package, 34/64 internal, 0/4 external>) (*DEFAULT-PATHNAME-DEFAULTS* . #P"") (*) (**) (***) (/) ...) ..)
28: ((SUBFUNCTION 1 SLYNK::SPAWN-CHANNEL-THREAD))
29: (MP::PROCESS-SG-FUNCTION 0 NIL NIL)
So it seems
(stream-element-type *standard-output*)is not setf'able?
Yes, http://www.lispworks.com/documentation/HyperSpec/Body/f_stm_el.htm does not mention that s-e-t is an accessor.
3: ((SUBFUNCTION 1 (METHOD STREAM:STREAM-WRITE-STRING (SLYNK-GRAY::SLY-OUTPUT-STREAM T))))
We need to figure out which stream this is and ensure that it is opened with character element type as well. Are you sure that all w-o-t-s calls in Slynk are patched to use character?
No, not at all. Just this one as mentioned by @joaotavora. The strategy it not clear. Do we patch all, or just the ones necessary?
Btw: the LW listener in IDE has default element-type set to character.
CL-USER 2 > (stream-element-type *standard-output*)
CHARACTER
Won't modifying the arguments to w-o-t-s below work better here?
Sure, also works.
we'll probably need to patch all w-o-t-s and open and with-open-file forms as a start and then check if we missed anything.
My idea is just to patch SLY's stream-using facilities that are directly concerned with sending such symbol names through the wire. It's indeed likely that there is more than just that part there. But unfortunately, I don't own a copy of Lispworks, so I can't test. (I was going to try with "lispworks personal edition" but then read http://www.lispworks.com/documentation/lw51/LWUG/html/lwuser-16.htm
This one is clearly not enough then, e.g. if we still get [error printing frame] in the debugger then whatever string streams the debugger uses still need patching. I have no idea how many of these exist in the slynk codebase.
(I was going to try with "lispworks personal edition" but then read http://www.lispworks.com/documentation/lw51/LWUG/html/lwuser-16.htm
You can still use the personal edition, load slynk into it via the listener, and then create a server and sly-connect to it.