logos
logos copied to clipboard
maciejhirsz
Create ridiculously fast Lexers
I'd like to help with the rewrite mentioned here: https://github.com/maciejhirsz/logos/releases/tag/v0.13 :) I already have some experience with logos' internals, at least the "frontend" part.
The following minimized example ```rust #[derive(Logos, Clone, Copy, Debug, PartialEq)] pub enum Token { #[token("a")] A, #[token("b")] B, #[regex(r"[ab]*c")] ABC, } fn main() { for (tok, span) in Token::lexer("aba").spanned() {...
Hello all, I'd like to open this discussion because, to me, it would be fascinating that **Logos** supports `Source` types others than `str` and `[u8]`, especially lazy readers like those...
When using a custom error type, is there a way to have a span provided with the error? what i mean is that a custom error type needs to satisfy...
Apparently, lexer behaviour differs when there are callbacks and when there aren't any. Specifically, when there aren't any callbacks, and there's an error at the start of the input, the...
This might be useful for #159, #324 and #340. I don't know anything about proc-macros in rust, but I'll try to implement it.
It would be useful if there was a built-in `bump_line()` fn that would bump the lexer to the next line. A more generic `bump_terminator(pat: P)` could be useful too. I...
Every single regex i could find uses either globals flags or lookaround, neither of which are supported by logos (to my knowledge, at least). Syntax like `[^(""")]` (NOT group of...
These are just some small cleanups I came across while working on the remaining changes for #222.
It would be really convenient to have an ability to inject custom EndOfInput token, just before the lexer starts to return None. ``` #[logos(error = LexerError)] #[logos(extras = LineTracker)] #[logos(skip...
