lark
lark copied to clipboard
Unify `Token` and `Tree` position information.
For trees the position information is placed inside the meta
attribute. When using Token
the information is directly stored in attributes. This makes it hard to write code which accepts Tree
and Token
. It would be good if there was an unified way to get this information.
I think it would make sense to add a meta
attribute to tokens as well. To keep backwards compatibility the direct attributes could be replaced with @property
to return the values from meta
.