lark icon indicating copy to clipboard operation
lark copied to clipboard

accepts() vs choices() in InteractiveParser

Open Daniel63656 opened this issue 1 month ago • 9 comments

I am using the InteractiveParser to feed tokens one at a time. I noticed this to be comparatively slow. After profiling it, I saw that a lot of copying is done. After checking the class, I noticed that accepts() (which I use in each step to query next allowed terminals) loops over all terminals returned from choices(), copies the parser state and then checks if feeding the terminal causes an UnexpectedToken exception. Why is this even necessary? The docs of choices() say: "only returns token types that are accepted by the current state" so why doing the very expensive copying of the entire parser state to verify this? Can I just use choices() and filter out the terminals?

Daniel63656 avatar May 21 '24 17:05 Daniel63656