logos
logos copied to clipboard
maciejhirsz
Create ridiculously fast Lexers
```rust #[derive(Logos, Clone, PartialEq, Debug)] pub enum Token { #[error] Error, // identifiers #[regex("[a-zA-Z_][a-zA-Z0-9_\\.]*", |lex| String::from(lex.slice()))] Id(String), // i'd like to skip spaces after function, space is an operator in...
I have a strange issue where [logos][1] fails tests only on Windows builds. [This][2] is how I define the token types. I test it across [this custom script][3]. While MacOS...
I'm working on a lexer for a language where I'd like to have `else` and `else if` lexed as separate tokens, but I'm running into suprising behaviour. In the following...
It would be neat if non-`'static` lifetimes could be used in extras. ```rust #[derive(Logos)] #[logos(extras = Option { #[error] Error, #[token("bar", |lex| lex.extras.unwrap())] Bar(&'a str), } ``` Currently, this fails...
I want to return `&str` of the underlying slice. ```rust fn remove_digit) -> Option { #[regex(r"\d[a-zA-Z]+", remove_digit)] WithNumber(&'a str), #[error] #[regex(r"[ \t\n\f\s]+", logos::skip, priority = 1000)] Error, } ``` Currently...
I'm making use of `Extras` to keep track of current line information, and in my parser I also want to `peek` the next token. The trouble is that if I...
I wonder if you would consider adding some syntactic sugar procedural macros for "case insensitive tokens" -- the way we have `#[token("foo")]`, if we want to match `Foo` and `FOO`...
At the moment errors are a single, opaque token. Is there a way to have more user-friendly, structured error values, with error recovery, instead of just returning `Token::Error`? It would...
Following [this conversation](https://github.com/maciejhirsz/logos/issues/126#issuecomment-619441936) from #126, it's possible to construct a regex that will produce a state machine graph which would never produce a match ~given that Logos never backtracks across...
when writing a regex using word boundaries I get this  Is there any plan to support something like this?
