Antlr4BuildTasks
Antlr4BuildTasks copied to clipboard
kaby76
Stack overflow in error listener
I've been putting this off, but there is a bug in the error reporter code I wrote for the Antlr program template. I'm checking in the code to reproduce it.
In order to compute the lookahead sets that a parse expects, the error listener makes a call back to the Antlr runtime to get the lookahead. If the token is invalid, round and round she goes.
pl1.dll!pl1.ErrorListener<int>.SyntaxError(System.IO.TextWriter output, Antlr4.Runtime.IRecognizer recognizer, int offendingSymbol, int line, int col, string msg, Antlr4.Runtime.RecognitionException e) Line 37
at C:\Users\kenne\Documents\AntlrExamples\pl1\ErrorListener.cs(37)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.ProxyErrorListener<int>.SyntaxError(System.IO.TextWriter output, Antlr4.Runtime.IRecognizer recognizer, int offendingSymbol, int line, int charPositionInLine, string msg, Antlr4.Runtime.RecognitionException e) Line 43
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\ProxyErrorListener.cs(43)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.Lexer.NotifyListeners(Antlr4.Runtime.LexerNoViableAltException e) Line 561
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\Lexer.cs(561)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.Lexer.NextToken() Line 178
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\Lexer.cs(178)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.BufferedTokenStream.Fetch(int n) Line 240
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\BufferedTokenStream.cs(240)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.BufferedTokenStream.Sync(int i) Line 220
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\BufferedTokenStream.cs(220)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.CommonTokenStream.LT(int k) Line 145
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\CommonTokenStream.cs(145)
pl1.dll!pl1.LASets.Compute(Antlr4.Runtime.Parser parser, Antlr4.Runtime.CommonTokenStream token_stream, int line, int col) Line 61
at C:\Users\kenne\Documents\AntlrExamples\pl1\LASets.cs(61)
pl1.dll!pl1.ErrorListener<int>.SyntaxError(System.IO.TextWriter output, Antlr4.Runtime.IRecognizer recognizer, int offendingSymbol, int line, int col, string msg, Antlr4.Runtime.RecognitionException e) Line 38
at C:\Users\kenne\Documents\AntlrExamples\pl1\ErrorListener.cs(38)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.ProxyErrorListener<int>.SyntaxError(System.IO.TextWriter output, Antlr4.Runtime.IRecognizer recognizer, int offendingSymbol, int line, int charPositionInLine, string msg, Antlr4.Runtime.RecognitionException e) Line 43
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\ProxyErrorListener.cs(43)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.Lexer.NotifyListeners(Antlr4.Runtime.LexerNoViableAltException e) Line 561
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\Lexer.cs(561)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.Lexer.NextToken() Line 178
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\Lexer.cs(178)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.BufferedTokenStream.Fetch(int n) Line 240
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\BufferedTokenStream.cs(240)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.BufferedTokenStream.Sync(int i) Line 220
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\BufferedTokenStream.cs(220)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.CommonTokenStream.LT(int k) Line 145
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\CommonTokenStream.cs(145)
pl1.dll!pl1.LASets.Compute(Antlr4.Runtime.Parser parser, Antlr4.Runtime.CommonTokenStream token_stream, int line, int col) Line 61
at C:\Users\kenne\Documents\AntlrExamples\pl1\LASets.cs(61)
pl1.dll!pl1.ErrorListener<int>.SyntaxError(System.IO.TextWriter output, Antlr4.Runtime.IRecognizer recognizer, int offendingSymbol, int line, int col, string msg, Antlr4.Runtime.RecognitionException e) Line 38
at C:\Users\kenne\Documents\AntlrExamples\pl1\ErrorListener.cs(38)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.ProxyErrorListener<int>.SyntaxError(System.IO.TextWriter output, Antlr4.Runtime.IRecognizer recognizer, int offendingSymbol, int line, int charPositionInLine, string msg, Antlr4.Runtime.RecognitionException e) Line 43
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\ProxyErrorListener.cs(43)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.Lexer.NotifyListeners(Antlr4.Runtime.LexerNoViableAltException e) Line 561
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\Lexer.cs(561)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.Lexer.NextToken() Line 178
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\Lexer.cs(178)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.BufferedTokenStream.Fetch(int n) Line 240
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\BufferedTokenStream.cs(240)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.BufferedTokenStream.Sync(int i) Line 220
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\BufferedTokenStream.cs(220)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.CommonTokenStream.LT(int k) Line 145
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\CommonTokenStream.cs(145)
pl1.dll!pl1.LASets.Compute(Antlr4.Runtime.Parser parser, Antlr4.Runtime.CommonTokenStream token_stream, int line, int col) Line 61
at C:\Users\kenne\Documents\AntlrExamples\pl1\LASets.cs(61)
pl1.dll!pl1.ErrorListener<int>.SyntaxError(System.IO.TextWriter output, Antlr4.Runtime.IRecognizer recognizer, int offendingSymbol, int line, int col, string msg, Antlr4.Runtime.RecognitionException e) Line 38
at C:\Users\kenne\Documents\AntlrExamples\pl1\ErrorListener.cs(38)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.ProxyErrorListener<int>.SyntaxError(System.IO.TextWriter output, Antlr4.Runtime.IRecognizer recognizer, int offendingSymbol, int line, int charPositionInLine, string msg, Antlr4.Runtime.RecognitionException e) Line 43
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\ProxyErrorListener.cs(43)
...
pl1.dll!pl1.LASets.Compute(Antlr4.Runtime.Parser parser, Antlr4.Runtime.CommonTokenStream token_stream, int line, int col) Line 61
at C:\Users\kenne\Documents\AntlrExamples\pl1\LASets.cs(61)
pl1.dll!pl1.ErrorListener<int>.SyntaxError(System.IO.TextWriter output, Antlr4.Runtime.IRecognizer recognizer, int offendingSymbol, int line, int col, string msg, Antlr4.Runtime.RecognitionException e) Line 38
at C:\Users\kenne\Documents\AntlrExamples\pl1\ErrorListener.cs(38)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.ProxyErrorListener<int>.SyntaxError(System.IO.TextWriter output, Antlr4.Runtime.IRecognizer recognizer, int offendingSymbol, int line, int charPositionInLine, string msg, Antlr4.Runtime.RecognitionException e) Line 43
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\ProxyErrorListener.cs(43)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.Lexer.NotifyListeners(Antlr4.Runtime.LexerNoViableAltException e) Line 561
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\Lexer.cs(561)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.Lexer.NextToken() Line 178
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\Lexer.cs(178)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.BufferedTokenStream.Fetch(int n) Line 240
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\BufferedTokenStream.cs(240)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.BufferedTokenStream.Sync(int i) Line 220
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\BufferedTokenStream.cs(220)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.BufferedTokenStream.Consume() Line 191
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\BufferedTokenStream.cs(191)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.Parser.Consume() Line 734
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\Parser.cs(734)
Antlr4.Runtime.Standard.dll!Antlr4.Runtime.Parser.Match(int ttype) Line 245
at C:\Users\kenne\Documents\antlr4\runtime\CSharp\runtime\CSharp\Antlr4.Runtime\Parser.cs(245)
pl1.dll!pl1_parserParser.pl1stmtlist(int _p) Line 749
at C:\Users\kenne\Documents\AntlrExamples\pl1\obj\Debug\netcoreapp3.1\pl1_parserParser.cs(749)
pl1.dll!pl1_parserParser.pl1pgm() Line 661
at C:\Users\kenne\Documents\AntlrExamples\pl1\obj\Debug\netcoreapp3.1\pl1_parserParser.cs(661)
pl1.dll!pl1.Program.Main(string[] args) Line 48
at C:\Users\kenne\Documents\AntlrExamples\pl1\Program.cs(48)
Correcting this problem is easy. But, there is another problem. It looks like I'm not getting any information of possible parses for an input with error (e.g., add after "PRTHDG1: PROCEDURE; asdf PUT FILE ...".
For sure, something is wrong with the interpreter because it faults very early on with no viable paths through the ATN. xx.txt x.txt
The problem is clearly the code in EnterState() to test whether this state needs to be revisited.
if (_visited.ContainsKey(new Pair<ATNState, int>(state, token_index)))
{
if (_log_parse)
{
System.Console.Error.WriteLine(
new String(' ', indent * 2)
+ "already visited.");
}
return null;
}
The code notices it's visiting state 284 (varnamequal) with input PSAM1. The first time it visited this state, it was in the context of a varnameref. The second time it is in the context of prestmtlist. Since it entered this state with the same input index, it assumes that this is just a loop within the ATN. But, it's not.
The problem with this code is that there is a different context between the time state was first visited and the second. State and token_index are insufficient. It should be the entire path to this point.
I'll hold off on a solution until I thought about this a little more, after a good hike tomorrow (Welch-Dickey).