Sean Hellum

See https://github.com/nushell/nushell/tree/master/crates/nu-parser

Can I contribute to the `Tokenizer` I really like this library

What still needs to be done? Can we create a draft PR?

Is there a branch where it is being worked on?

So what is the idea behind the Tokenizer so that I can get started

So basically a lexer?

I'm trying to highlight keywords typically I would do \bkeyword\b right now keywordfoo highlights keyword and it should not highlight at all EDIT: To demonstrate 

OK, I read the blog and added a new regex. FYI The lexer expands to (using `cargo expand`) ```rust enum TheLexer { #[end] End, #[error] Error, #[token = " "]...

How would I translate this ```rust while tokens.token != $enumName::End ``` to 0.11

i.e. how can I check the current token before I would use tokens.token