toki
toki copied to clipboard
Published
20 hours ago •
taylorchu
taylorchu
general tokenizer/scanner in Go
toki
Regexp tokenizer/scanner in Go.
The scanner takes in a list of token definitions and string input. it is general and easy to use.
token definition
a token is a pair of constant and regexp string.
taylorchu