Simon Krajewski

Yes, this has been a problem ever since rulesets were added. Originally the problem was that the lexer was based on a data representation which did not support rewinding. I...

This would mean disabling the token cache completely, which I don't think is viable performance-wise. The lexer shouldn't have to tokenize the same thing twice if not necessary.

I'll see what I can do about the caching issue soon. Out of curiosity, what are you trying to parse?

The problem is this: - You fetch a token with ruleset A. - The parser caches it. - You don't consume it. - Instead you try to fetch a token...

I'm not sure what that would look like unless you want to call peek and junk manually all the time, which I think you already can anyway. One thing we...

I don't really trust macro-in-macro anyway, but that should be possible. It would look a bit weird because you would have to pass the switch expression as an argument to...

It's an easy change because all the code is already there, so I have added hxparse.Parser.parse which expects a switch expression as argument. Please check if that works for you.

@fussybeaver: We'll have to move that parse method somewhere else. I forgot that Parser is supposed to be `@:generic` and that disallows static fields, even macros.

Actually I might fix it in core Haxe: https://github.com/HaxeFoundation/haxe/issues/3766

Back to the original issue, here's a small example to reproduce it: ``` haxe import byte.ByteData; import hxparse.*; class MyLexer extends hxparse.Lexer implements RuleBuilder { static public var tok =...