lark icon indicating copy to clipboard operation
lark copied to clipboard

Standalone parser is not deterministic

Open Kyuuhachi opened this issue 9 months ago • 2 comments

Describe the bug

I want to keep a standalone parser in my repo, to make it easy to use without any dependencies. But even rebuilding exactly the same grammar produces a different parse table, which makes git diff unhappy. I presume this is because set() is unordered.

To Reproduce

The simplest possible grammar: an empty start rule, start:. Then compile it twice and compare the output, diff =(python -m lark.tools.standalone a.g) =(python -m lark.tools.standalone a.g).

3157c3157
< ... 'parser': {'tokens': {0: 'start', 1: '$END'}, 'states': {0: {}, 1: {0: (0, 0), 1: (1, {'@': 0})}}, 'start_states': {'start': 1}, 'end_states': {'start': 0}}, ...
---
> ... 'parser': {'tokens': {0: 'start', 1: '$END'}, 'states': {0: {0: (0, 1), 1: (1, {'@': 0})}, 1: {}}, 'start_states': {'start': 0}, 'end_states': {'start': 1}}, ...

(I cut off some irrelevant parts)

Kyuuhachi avatar Sep 22 '23 15:09 Kyuuhachi