logos
logos copied to clipboard
maciejhirsz
Create ridiculously fast Lexers
Basically, I was wondering if it's possible to have it so that when a piece of text doesn't match any other tokens, it is assigned a default token. For example:...
Not really sure what's wrong with my regexes. I tested them in Python and they worked fine, so I'm pretty sure they're valid under the rules of the regex crate...
I am writing parser generators and would like to experiment with using logos to create lexical scanners, which should be faster than the ones that I've coded. Before I proceed,...
Logos: v0.12.1 Rustc: 1.64.0 Linux NixOS 5.15.82 #\1-NixOS SMP Thu Dec 8 10:28:45 UTC 2022 x86_64 GNU/Linux ```rs use logos::Logos; #[derive(Logos, Debug, PartialEq)] enum Tok { #[token("other")] Other, #[error] ERROR,...
Is it possible to implement lexer over a segement input (E.g. RopeSlice) ? I found that there is a `Source` trait in `logos`, but I have no idea how to...
Hello! For the context, I am writing a TeX file parser, and I use `Logos==0.12.1` to create a lexer. You can find a (reduced) version of my `Token` implementation: ```rust...
This code: ```rust use logos::Logos; // logos 0.12.1 #[derive(Logos)] enum Token { #[regex("x()*")] What, #[error] Error, } ``` Yields this compile error: ``` error: proc-macro derive panicked --> src\lib.rs:3:10 |...
I have a doc comment token, that uses the following regexes: - `///[^\n]*` - `/\*\*([^\*]|\*[^\*/]|\*\*[^/])*\*?\*/` As well as a regular comment token, that skips the following regexes: - `//[^\n]*` -...
Hi, I accidentally messed up one of my regexes and the binary that uses Logos started crashing. In debug it fails with stack overflow, in debug with GDB with a...
Fixes #385 This removes unreachable branches from LUTs in the generated code, so that LLVM can see that the error case is unreachable for lexers like in the linked issue
