ANTLR3 C target runs into infinite loop for input starting with NULL byte

[mjakubicek]

I'm not sure whether it is a lexer or parser issue, but giving an input starting with a NULL byte to be parsed by the attached grammar result into an infinite loop.

Looking at the looping process with gdb yields the following stack trace:

0x00000037d7011bf8 in get (ts=<optimized out>, i=1592905172)
    at src/antlr3tokenstream.c:503
503         return  (pANTLR3_COMMON_TOKEN)(cts->tokens->get(cts->tokens, i));  /* Token index is zero based but vectors are 1 based */
(gdb) up

#1  0x00000037d7012100 in toStringSS (ts=0x1248320, start=<optimized out>, 
    stop=4294967295) at src/antlr3tokenstream.c:566
566                 tok = ts->get(ts, i);
(gdb) 
#2  0x000000000040c647 in throwEvalQueryException (recognizer=0x12485a0, 
    tokenNames=0x6a9500 <cqpParserTokenNames>) at cqpeval.cc:200
200             ttext       = parser->tstream->toStringTT(parser->tstream, theToken, theToken);
(gdb) 
#3  0x0000000000418760 in seq (ctx=ctx@entry=0x12862f0) at cqpParser.cc:3850
3850                    PREPORTERROR();
(gdb) 
#4  0x00000000004189c0 in sequence (ctx=ctx@entry=0x12862f0) at cqpParser.cc:3559
3559                seq55=seq(ctx);
(gdb) 
#5  0x0000000000418f90 in query (ctx=0x12862f0) at cqpParser.cc:1007
1007                sequence1=sequence(ctx);
(gdb) 
#6  0x000000000040bf4c in eval_cqpquery (query=<optimized out>, corp=corp@entry=
    0x7fff44f74070) at cqpeval.cc:47
47          cqpParser_query_return cqpAST = parser->query(parser);

[mjakubicek]

The grammar can be found here: https://downloads.sketchengine.co.uk/cqp.g