eudoxia0
Testsuite fails
For example, with SBCL 2.0.1.debian I have the following on the current master branch (commit fc35b2a)
(pushnew "/path/to/docparser/" asdf:*central-registry* :test #'equal)
(ql:quickload :docparser-test)
(fiveam:run! 'docparser-test::tests)
Did 80 checks.
Pass: 75 (93%)
Skip: 0 ( 0%)
Fail: 5 ( 6%)
Failure Details:
--------------------------------
CFFI-NODES in TESTS []:
Unexpected Error: #<SB-KERNEL:INDEX-TOO-LARGE-ERROR expected-type:
(INTEGER 0 22)
datum: 33>
Invalid index 33 for (VECTOR T 32) with fill-pointer 23, should be a non-negative integer below 23..
--------------------------------
--------------------------------
METHOD-NODES in TESTS []:
Unexpected Error: #<SB-KERNEL:INDEX-TOO-LARGE-ERROR expected-type:
(INTEGER 0 22)
datum: 28>
Invalid index 28 for (VECTOR T 32) with fill-pointer 23, should be a non-negative integer below 23..
--------------------------------
--------------------------------
TYPE-NODES in TESTS []:
'DOCPARSER:STRUCT-NODE
evaluated to
DOCPARSER:STRUCT-NODE
which is not
TYPEP
to
--------------------------------
--------------------------------
TYPE-NODES in TESTS []:
"REC2"
evaluated to
"REC2"
which is not
STRING=
to
"PRINTF"
--------------------------------
--------------------------------
TYPE-NODES in TESTS []:
Unexpected Error: #<SB-KERNEL:INDEX-TOO-LARGE-ERROR expected-type:
(INTEGER 0 22)
datum: 23>
Invalid index 23 for (VECTOR T 32) with fill-pointer 23, should be a non-negative integer below 23..
--------------------------------
For CCL 1.12.1 (v1.12.1) LinuxX8664 the same snipped produces:
Did 230 checks.
Pass: 220 (95%)
Skip: 0 ( 0%)
Fail: 10 ( 4%)
Failure Details:
--------------------------------
CFFI-NODES in TESTS []:
'DOCPARSER:CFFI-TYPE
evaluated to
DOCPARSER:CFFI-TYPE
which is not
TYPEP
to
#<macro printf (CONTROL &REST VARARGS12399)>
--------------------------------
--------------------------------
CFFI-NODES in TESTS []:
"SIZE-T"
evaluated to
"SIZE-T"
which is not
STRING=
to
"PRINTF"
--------------------------------
--------------------------------
CFFI-NODES in TESTS []:
'DOCPARSER:CFFI-ENUM
evaluated to
DOCPARSER:CFFI-ENUM
which is not
TYPEP
to
--------------------------------
--------------------------------
CFFI-NODES in TESTS []:
"NUMS"
evaluated to
"NUMS"
which is not
STRING=
to
"CUNION"
--------------------------------
--------------------------------
CFFI-NODES in TESTS []:
Unexpected Error: #<CCL:NO-APPLICABLE-METHOD-EXISTS #x302002038B9D>
There is no applicable method for the generic function:
#<STANDARD-GENERIC-FUNCTION DOCPARSER:CFFI-ENUM-VARIANTS #x3020019888BF>
when called with arguments:
().
--------------------------------
--------------------------------
METHOD-NODES in TESTS []:
'DOCPARSER:MACRO-NODE
evaluated to
DOCPARSER:MACRO-NODE
which is not
TYPEP
to
#<method test-method ((TC TEST-CLASS) A)>
--------------------------------
--------------------------------
METHOD-NODES in TESTS []:
"INDIRECTLY-DEFINE-FUNCTION"
evaluated to
"INDIRECTLY-DEFINE-FUNCTION"
which is not
STRING=
to
"TEST-METHOD"
--------------------------------
--------------------------------
METHOD-NODES in TESTS []:
0
evaluated to
0
which is not
=
to
2
--------------------------------
--------------------------------
METHOD-NODES in TESTS []:
'DOCPARSER:FUNCTION-NODE
evaluated to
DOCPARSER:FUNCTION-NODE
which is not
TYPEP
to
#<macro indirectly-define-function NIL>
--------------------------------
--------------------------------
METHOD-NODES in TESTS []:
"HIDDEN-FUNCTION"
evaluated to
"HIDDEN-FUNCTION"
which is not
STRING=
to
"INDIRECTLY-DEFINE-FUNCTION"
--------------------------------
--------------------------------
So apparently, the order of nodes in the parsed index differs depending on the lisp implementation (due to different expansion of standard CL macros?).
I imagine this is because of how docparser works, because macroexpansion is unreliable. It's not great.
At first sight the public API does not expose any order between nodes. If so the test suite approach which fails currently - expecting the desired doc-parser:node instances to be located at specific integer positions in the index - be adjusted. Instead, the tests could find the nodes using the query function, and then validate expected properties of the nodes (name, docstring, lambda list, etc).
Do you think that approach is appropriate?
Yes, making the tests depend less on implementation-specific behaviour and be more reliable would be good, but I don't have the time or particularly the desire to make any changes. But I will review PRs.