Olivier Chafik

@liljohnak ANTLR generates parsers that operate on a given string and backtrack when they picked the wrong branch if/when the grammar is ambiguous. In contrast, llama.cpp's grammar constrained sampling (https://github.com/ggerganov/llama.cpp/pull/1773)...

> Please mind to push commits on your fork first as it triggers lot of CI runs on the main repo. @phymbert sorry for the CI noise again today, wanted...

> `llama_grammar_element wouldn't hold a rule ID or a unicode code point, but instead the number of times that the previous rule should be repeated. @HanClinto I did wonder about...

> So something like "a"{2,5} would hydrate to something like: (essentially "a"{2} ("a"{3} | )) > `{LLAMA_GRETYPE_CHAR, 'a'},` > `{LLAMA_GRETYPE_REPEAT_N, 2},` > `{LLAMA_GRETYPE_CHAR, 'a'},` > `{LLAMA_GRETYPE_REPEAT_N, 3},` > `{LLAMA_GRETYPE_ALT, 0},`...

> Okay -- I think that's probably enough of an encouragement for me to at least try taking a stab at it this weekend. Yayyy, good luck! > I don't...

> > (Ugh, this one and its vectors just look like a ripe fractal fruit looking deceitfully low-hanging, yet a bit elusive) > Yeah, seriously. I follow the code mentally...

@HanClinto I did a first bit of benchmarking / comparison of outputs on @AlienKevin's grammar from https://github.com/ggerganov/llama.cpp/issues/4218#issuecomment-1836540046 on this PR and it seems to be ~~8% faster~~ (**edit**) 5% slower...

@HanClinto did you get a chance to fiddle w/ an alternative repetition implementation? I'm itching to try something tonight tbh :-D

> I'm helping coach a middle-school robotics team and they have their big competition for the next 3 days starting tomorrow 🤞🤖🦾

@HanClinto I've taken a (permanent?) break from the alternative repetition implementation front. Instead, I've updated the grammar parser's rewrites to match more closely the existing + / ? / *...