Interlisp
WIP: make variable types in simulate_unbind match OP_unbind
Just posting looking for ssome help. When I stepped through the code in values / simulate_unbind, there were some cases where "value" was a negative int instead of a LispPTR*. So I tried (cargo cult programming) making simulate_unbind look like OP_unbind, renaming variables and declarations to match. I left the MAKEFREEBLOCK commented out.
Did you look at where you are changing the signedness of the variables that are being right-shifted? When you go from signed to unsigned you will lose the sign-extension -- are you sure that that is what is needed?
The code for the unbind implementation isn't N_OP_unbind, it's the code in inlineC.h --
#define UNBIND \
do { \
register int num; \
register LispPTR *ppvar; \
register int i; \
register LispPTR value; \
for (; (((int)*--CSTKPTRL) >= 0);) \
; \
value = *CSTKPTR; \
num = (~value) >> 16; \
ppvar = (LispPTR *)((DLword *)PVAR + 2 + GetLoWord(value)); \
for (i = num; --i >= 0;) { *--ppvar = 0xffffffff; } \
nextop1; \
} while (0)
I'm doing this since i'm not sure what's going on. The unbind count should be reset to 0 each RETURN scanned and stackframe. Each frame popped off by values OR valueslist should be freed?
Yeah, I've seen way too much cargo-cult programming, mostly on the VLC dev list, where the practitioners are often roundly excoriated for it.
I'm not sure that the problem here is actually in the simulate_unbind() or at least not in the difference from the UNBIND opcode.
The BIND/UNBIND in the regular opcode stream aren't making new stack frames, are they? They should be stuffing a value into, or marking as unused, slots that are in the current frame.
My understanding from the code is that the MISCN/values code, where it does
caller->pc = (UNSIGNED)pc + FN_OPCODE_SIZE - (UNSIGNED)fnhead;
is simulating the return of the lower procedure and skipping back to where the \MVLIST would have returned to, if it were actually being called, and if there were any UNBIND operations between here and there, the PVAR entries in those frames need to be unbound (i.e., have the binding smashed to the unbound mark).
So, if the stack isn't being properly set up it's not a problem in the simulated unbind, but more likely in the way the return is being faked when the PC of the caller is adjusted, and the lower procedure's stack frame isn't being freed.
Rereading, a question: shouldn't it only need to simulate unbinds in frames that will still exist after we fake the return? Stack frames that (should) have been released don't need to have adjustments made.
I've annotated what I think is going on here -- the "outer" code is stackbug, the indented is stkbug1. Does this make more sense to you now? I think it clears it up quite a bit for me:
0: 2C stkmin
2: 0 na
4: 0 pv
6: 30 startpc
8: 60 byteswapped: NIL argtype: 0
A: E340 frame name: STACKBUG
C: 8 ntsize
E: 200 nlocals: 2 fvaroffset: 0
name table:
10: 0 60 E2 FA 20: PVAR 1: A0023
14: 0 0 3 F9 24: PVAR 0: CHARCODE
18: 0 0 0 0 28:
1C: 0 0 0 0 2C:
----
30: 11 10 0 BIND ; CHARCODE
33: 67 0 60 E3 36 ACONST STKBUG1
38: 11 1 1 BIND A0023; binding PVAR 1
3B: 69 'T
3C: 6B '1
3D: 49 PVAR A0023
3E: E APPLYFN calls STKBUG1, inline here
----
18: 6A '0
19: 6C A SIC 10
1B: A 0 0 A F FN2 RAND
20: 6A '0
21: F0 EQ
22: 6C 2A SIC 42
24: 24 1 2 MISCN values skips this opcode
27: 10 RETURN notes return, gets caller from CLINK
28: 0 -X- positions scan pc to #3F
----
3F: 12 UNBIND #3F: and finds UNBIND opcode
40: 9 0 0 3 5D FN1 \MVLIST then finds FN1 \MVLIST, notices that if we returned to #45
we would have skipped an UNBIND, so we must simulate it,
create the result, and fake a return to #45. The stack frame for
STKBUG1 should be released, but that has nothing to do with unbind?
45: 1 CAR
46: 58 PVAR_ CHARCODE
47: B3 EC TJUMPX-> 33
49: 69 'T
4A: 10 RETURN
4B: 0 -X-
The next thing to investigate is exactly how it implements the return from STKBUG1 to the point following the \MVLIST... does it actually execute a RETURN opcode to clean up the stack frames? Is it faked and thus (bugily) missing a necessary stack adjustment?
the problem only happens when returning multiple values to an UNBIND followed by a FN1 \MVLIST. No UNBIND, no problem. Would it help to try it on a 32-bit or Big-endian arch machine?
I did that long ago. It makes no difference whether it's 32- or 64- bit, or little- or big-endian.
Even if simulate_unbind() does nothing but print the stack pointer and return... the stack pointer passed is increasing by 4 bytes each time, and I don't think that that is correct. I surmise that the problem is not in simulate_unbind().
In the case we are having trouble with, the way it returns from STKBUG1 is to just continue execution after the MISCN opcode, executing the RETURN opcode, however at that point the saved PC in the frame belonging to STACKBUG has been adjusted so that it skips over the UNBIND and FN1 \MVLIST and continues normal execution at the CAR opcode. It doesn't matter whether you execute simulate_unbind() or not, you get the same results, and the stack grows by 4 bytes at each call.
Here's an execution trace (slightly modified from the default, opcodes in hex), and where STKBUG1 has just SIC/SIC/MISCN/
PC= 0x10a5bc0 (fn(10a5b90)+0x30) op= 11 TOS= 0x8c3 START OF STACKBUG
PC= 0x10a5bc3 (fn(10a5b90)+0x33) op= 67 TOS= 0xfffe0000
PC= 0x10a5bc8 (fn(10a5b90)+0x38) op= 11 TOS= 0x60e336
PC= 0x10a5bcb (fn(10a5b90)+0x3b) op= 69 TOS= 0xfffe0002
PC= 0x10a5bcc (fn(10a5b90)+0x3c) op= 6b TOS= 0x4c
PC= 0x10a5bcd (fn(10a5b90)+0x3d) op= 49 TOS= 0xe0001
PC= 0x10a5bce (fn(10a5b90)+0x3e) op= 0e TOS= 0x60e336 APPLY
PC= 0x10f781c (fn(10f7800)+0x1c) op= 6c TOS= 0x4c SIC 33
PC= 0x10f781e (fn(10f7800)+0x1e) op= 6c TOS= 0xe0021 SIC 42
PC= 0x10f7820 (fn(10f7800)+0x20) op= 24 TOS= 0xe002a MISCN
OP_miscn: enter PC 0x10f7820 CurrentStackPTR 0x4aab50 arg_count 2
OP_miscn: exit PC 0x10f7823 CurrentStackPTR 0x4aab4c stk 0x4aab50
PC= 0x10f7823 (fn(10f7800)+0x23) op= 10 TOS= 0x6257b6 RETURN
PC= 0x10a5bd5 (fn(10a5b90)+0x45) op= 01 TOS= 0x6257b6 CAR
PC= 0x10a5bd6 (fn(10a5b90)+0x46) op= 58 TOS= 0xe0021 PVAR_0
PC= 0x10a5bd7 (fn(10a5b90)+0x47) op= b3 TOS= 0xe0021 TJUMPX->33
PC= 0x10a5bc3 (fn(10a5b90)+0x33) op= 67 TOS= 0xfffe0002 ACONST 'STKBUG1
PC= 0x10a5bc8 (fn(10a5b90)+0x38) op= 11 TOS= 0x60e336 BIND
PC= 0x10a5bcb (fn(10a5b90)+0x3b) op= 69 TOS= 0xfffe0002 T
PC= 0x10a5bcc (fn(10a5b90)+0x3c) op= 6b TOS= 0x4c '1
PC= 0x10a5bcd (fn(10a5b90)+0x3d) op= 49 TOS= 0xe0001 PVAR
PC= 0x10a5bce (fn(10a5b90)+0x3e) op= 0e TOS= 0x60e336 APPLY
PC= 0x10f781c (fn(10f7800)+0x1c) op= 6c TOS= 0x4c SIC 33
PC= 0x10f781e (fn(10f7800)+0x1e) op= 6c TOS= 0xe0021 SIC 42
PC= 0x10f7820 (fn(10f7800)+0x20) op= 24 TOS= 0xe002a MISCN
OP_miscn: enter PC 0x10f7820 CurrentStackPTR 0x4aab54 arg_count 2 but CurrentStackPTR has moved
OP_miscn: exit PC 0x10f7823 CurrentStackPTR 0x4aab50 stk 0x4aab54
PC= 0x10f7823 (fn(10f7800)+0x23) op= 10 TOS= 0x6257b4
PC= 0x10a5bd5 (fn(10a5b90)+0x45) op= 01 TOS= 0x6257b4
PC= 0x10a5bd6 (fn(10a5b90)+0x46) op= 58 TOS= 0xe0021
PC= 0x10a5bd7 (fn(10a5b90)+0x47) op= b3 TOS= 0xe0021
PC= 0x10a5bc3 (fn(10a5b90)+0x33) op= 67 TOS= 0xfffe0002
PC= 0x10a5bc8 (fn(10a5b90)+0x38) op= 11 TOS= 0x60e336
PC= 0x10a5bcb (fn(10a5b90)+0x3b) op= 69 TOS= 0xfffe0002
PC= 0x10a5bcc (fn(10a5b90)+0x3c) op= 6b TOS= 0x4c
PC= 0x10a5bcd (fn(10a5b90)+0x3d) op= 49 TOS= 0xe0001
PC= 0x10a5bce (fn(10a5b90)+0x3e) op= 0e TOS= 0x60e336
PC= 0x10f781c (fn(10f7800)+0x1c) op= 6c TOS= 0x4c
PC= 0x10f781e (fn(10f7800)+0x1e) op= 6c TOS= 0xe0021
PC= 0x10f7820 (fn(10f7800)+0x20) op= 24 TOS= 0xe002a
OP_miscn: enter PC 0x10f7820 CurrentStackPTR 0x4aab58 arg_count 2
OP_miscn: exit PC 0x10f7823 CurrentStackPTR 0x4aab54 stk 0x4aab58
PC= 0x10f7823 (fn(10f7800)+0x23) op= 10 TOS= 0x6257b2
PC= 0x10a5bd5 (fn(10a5b90)+0x45) op= 01 TOS= 0x6257b2
PC= 0x10a5bd6 (fn(10a5b90)+0x46) op= 58 TOS= 0xe0021
PC= 0x10a5bd7 (fn(10a5b90)+0x47) op= b3 TOS= 0xe0021
PC= 0x10a5bc3 (fn(10a5b90)+0x33) op= 67 TOS= 0xfffe0002
PC= 0x10a5bc8 (fn(10a5b90)+0x38) op= 11 TOS= 0x60e336
PC= 0x10a5bcb (fn(10a5b90)+0x3b) op= 69 TOS= 0xfffe0002
PC= 0x10a5bcc (fn(10a5b90)+0x3c) op= 6b TOS= 0x4c
PC= 0x10a5bcd (fn(10a5b90)+0x3d) op= 49 TOS= 0xe0001
PC= 0x10a5bce (fn(10a5b90)+0x3e) op= 0e TOS= 0x60e336
PC= 0x10f781c (fn(10f7800)+0x1c) op= 6c TOS= 0x4c
PC= 0x10f781e (fn(10f7800)+0x1e) op= 6c TOS= 0xe0021
PC= 0x10f7820 (fn(10f7800)+0x20) op= 24 TOS= 0xe002a
OP_miscn: enter PC 0x10f7820 CurrentStackPTR 0x4aab5c arg_count 2
OP_miscn: exit PC 0x10f7823 CurrentStackPTR 0x4aab58 stk 0x4aab5c
PC= 0x10f7823 (fn(10f7800)+0x23) op= 10 TOS= 0x6257ae
PC= 0x10a5bd5 (fn(10a5b90)+0x45) op= 01 TOS= 0x6257ae
PC= 0x10a5bd6 (fn(10a5b90)+0x46) op= 58 TOS= 0xe0021
PC= 0x10a5bd7 (fn(10a5b90)+0x47) op= b3 TOS= 0xe0021
PC= 0x10a5bc3 (fn(10a5b90)+0x33) op= 67 TOS= 0xfffe0002
and again, with the stack pointer included in the trace -- so what opcode is not having the expected effect on the stack (this notes stack pointer at the start of the opcode)
PC= 0x1085bc0 (fn(1085b90)+0x30) op= 11 TOS= 0x8c3 STACK 0x48ab1c BIND ; CHARCODE
PC= 0x1085bc3 (fn(1085b90)+0x33) op= 67 TOS= 0xfffe0000 STACK 0x48ab20 ACONST 'STKBUG1 (push)
PC= 0x1085bc8 (fn(1085b90)+0x38) op= 11 TOS= 0x60e336 STACK 0x48ab24 BIND A0023
PC= 0x1085bcb (fn(1085b90)+0x3b) op= 69 TOS= 0xfffe0002 STACK 0x48ab24 'T (push)
PC= 0x1085bcc (fn(1085b90)+0x3c) op= 6b TOS= 0x4c STACK 0x48ab28 '1 (push)
PC= 0x1085bcd (fn(1085b90)+0x3d) op= 49 TOS= 0xe0001 STACK 0x48ab2c PVAR A0023 (push)
PC= 0x1085bce (fn(1085b90)+0x3e) op= 0e TOS= 0x60e336 STACK 0x48ab30 APPLYFN
PC= 0x10d781c (fn(10d7800)+0x1c) op= 6c TOS= 0x4c STACK 0x48ab4c SIC 33 <<<< (push)
PC= 0x10d781e (fn(10d7800)+0x1e) op= 6c TOS= 0xe0021 STACK 0x48ab50 SIC 42 (push)
PC= 0x10d7820 (fn(10d7800)+0x20) op= 24 TOS= 0xe002a STACK 0x48ab54 MISCN (pop)
OP_miscn: enter PC 0x10d7820 CurrentStackPTR 0x48ab50 arg_count 2
OP_miscn: exit PC 0x10d7823 CurrentStackPTR 0x48ab4c stk 0x48ab50
PC= 0x10d7823 (fn(10d7800)+0x23) op= 10 TOS= 0x6257a4 STACK 0x48ab50 RETURN
PC= 0x1085bd5 (fn(1085b90)+0x45) op= 01 TOS= 0x6257a4 STACK 0x48ab28 CAR
PC= 0x1085bd6 (fn(1085b90)+0x46) op= 58 TOS= 0xe0021 STACK 0x48ab28 PVAR_ CHARCODE
PC= 0x1085bd7 (fn(1085b90)+0x47) op= b3 TOS= 0xe0021 STACK 0x48ab28 TJUMPX -> 33
PC= 0x1085bc3 (fn(1085b90)+0x33) op= 67 TOS= 0xfffe0002 STACK 0x48ab24 ACONST 'STKBUG1 (push)
PC= 0x1085bc8 (fn(1085b90)+0x38) op= 11 TOS= 0x60e336 STACK 0x48ab28
PC= 0x1085bcb (fn(1085b90)+0x3b) op= 69 TOS= 0xfffe0002 STACK 0x48ab28
PC= 0x1085bcc (fn(1085b90)+0x3c) op= 6b TOS= 0x4c STACK 0x48ab2c
So when we arrive back at STACKBUG+0x33, we've got 4 more bytes on the stack than when we started.
it's the missing FN1 \MVLIST, which SHOULD pop the stack with its parameter and PUSH the return value. Because it incrememts the PC so the FN1 \MVLIST doesn't happen, the stack is 1 too big. The only puzzle is why it doesn't stack overflow more often.
Wouldn't the combination of POP the parameter and PUSH the return value be stack-neutral, so no adjustment should be necessary? What am I missing? I don't think an argument got pushed for \MVLIST, and the result of the MISCN should (?) have been on the stack as the argument for CAR...?
The call decrements the stack and the return pushes the return value. Yes, FN1 is neutral.
I've been thinking of ways of doing multiple-value-return in a way that is straightforward but doesn't require mass recompiling / macro changes. I have an idea but it requires changing one Lisp function, \MVLIST as well as maiko -- a coordinated change.
The way it's set up now, there are to opcodes, Values and ValuesList. Both return one value (the first for Values and CAR of the list for ValuesList) UNLESS the next opcode is (a) a call to \MVLIST or a RETURN, JUMP, or UNBIND (being the only cases that pass along multiple values). In those cases it keeps on looking for \MVLIST, following up the control stack or the jumps.
There are 4 cases: the first is ordinary return to ordinary returnee, MV return to MVLIST returnee , MV return to 1-value returnee, ordinary return to MVLIST returnee.
in case 2 it finds the returnees MVLIST call and forces a return with the PC set to skip the call to \MVLIST. In case 3 it doesn't find a MVLIST call and just returns the single value as if cl:values was PROG1. In case 4, you wind up calling \MVLIST which is just (lambda (x) (list x)).
The solution I was going to propose was not to modify the PC or frame pointer or simulate any unbinds, but instead pass on a return value (cons '\mvlist values-list) in the case where a fn1 \mvlist was found. at the same change \mvlist (lambda (x) (cond ((and (listp x) (eq (car x) '\mvlist)) (cdr x)) (t (list x))))) no simulate, no slow return, no changing the PC on the stack.
But now that I've written this, i want to go back and check to see if simulate_unbind properly pops the bind mark off the stack (if I can remember what BIND does).
simulate_unbind (and the unbind opcode itself) don't pop the bind mark off the stack, or manipulate the stack pointer at all, they just smash the binding slot with a value that indicates unbound.