Vadim Markovtsev

Node positions are not influenced by the token values, are they.

Yep tree positions matter. I need the explicit hashes, `Equal` does not help with going from O(n^2) to O(n)

@dennwc @creachadair I wouldn't really close this one. Although I do understand the implemented logic, 1. Somebody has to calculate the "correct" node spans and currently, it is us, the...

Actually, we are likely to use the `Annotated` mode in our current analyzers, because we need to reconstruct the original token stream byte-to-byte.

The current workaround is simple: I look at the difference between `file_contents[start_position.offset:end_position.offset]` and Token and record prefixes and suffixes.

This is unexpected for me as well.

Hey, thanks for testing the library. No, I haven't seen this yet. Can you reproduce it in a Dockerfile?

@tims0 which model type do you mean exactly? We provide sample code how to load the models, but using them requires understanding of the goal.

@marnovo What is more structure in the descriptions? The process of training the models is deprecated and does not use UAST and Babelfish for example. In fact, they are hacky...