Peter Blackson

> a way to make tokens that span multiple input positions. Ah yes, I expected you to mention multiple input positions, just without certainty what they might be useful for....

> Sorry if I've misunderstood you, but I'm pretty sure that if you've implemented Earley, you've chosen both. As far as I can tell dynamic programming is inherent to the...

> Are you going all the way to CNF? Almost. I don't care that much where my symbols are terminal or nonterminal.

My RHS is always 1 or 2 symbols - binarized, just like CNF. Only Start can be nulling, just like CNF. One terminal and one nonterminal may appear on RHS,...

> in-progress parses were still needed at the end to recover the relationship between intermediate dot positions and input position. But if you can store the token IDs at the...

- Much easier to eliminate nulling symbols after binarization. Just like in Chomksy Normal Form. There are only 4 choices per rule, I called them: `enum BinarizedRhsRange { All, Left,...

> Early/Leo items in 1.5 64-bit words Interesting, what do you store in there?

> what's the motivation for breaking rules down Ah, I recall it now. The main motivation is constructing a **binarized** bocage (Shared Packed Parse Forest) to avoid exponential blowup.

> I don't think I understand why binarizing would make a difference in [no duplicated in-progress items] As long as completion is invoked once per (origin, symbol predicted there) we...

> Most of the item storage is described here So, instead of the bocage node ~~ref~~ id, you have a mainstem. Also, you have the LHS symbol which I don't...