terminal icon indicating copy to clipboard operation
terminal copied to clipboard

New tokenization & integer parsing helpers

Open lhecker opened this issue 1 year ago • 3 comments

Reading through our existing patterns for integer parsing, I noticed that we'd be better off returning them as optionals. This also allowed me to improve the implementation to support integers all the way up to their absolute maximum/minimum.

Furthermore, I noticed that prefix_split was unsound: If the last needle character was the last character in the remaining text, the remaining text would be updated to an empty string view. The caller would then have no idea if there's 1 more token left or if the string is truly empty. To solve this, this PR introduces an iterator class. This will allow it to be used in our VT parser code.

lhecker avatar Sep 25 '24 18:09 lhecker