futaba icon indicating copy to clipboard operation
futaba copied to clipboard

Prevent attempts at nesting character sets in regex filter

Open JoshuaS3 opened this issue 1 year ago โ€ข 0 comments

Not sure how I didn't catch this when testing the initial regex feature a couple years ago. Basically, nesting character sets is impossible in Python's regex, which is what my filter code tried to do because it wasn't checking for IN tokens, so those confusable glyphs would be discarded by the regex compiler (thankfully it doesn't outright throw an error). This PR adds that check and performs confusable insertion in place for those sets. Also cleans up some of the code in that area.

Example of the issue: h[ea]lp would effectively be converted into this, which is invalid regex:

[h๏ฝˆโ„Ž๐ก๐’‰๐’ฝ๐“ฑ๐”ฅ๐•™๐–๐—๐—ต๐˜ฉ๐™๐š‘าปีฐแ‚]**[**[eโ„ฎ๏ฝ…โ„ฏโ…‡๐ž๐‘’๐’†๐“ฎ๐”ข๐•–๐–Š๐–พ๐—ฒ๐˜ฆ๐™š๐šŽ๊ฌฒะตาฝ][aโบ๏ฝ๐š๐‘Ž๐’‚๐’ถ๐“ช๐”ž๐•’๐–†๐–บ๐—ฎ๐˜ข๐™–๐šŠษ‘ฮฑ๐›‚๐›ผ๐œถ๐ฐ๐žชะฐ]**]**[l๐Ÿ๐Ÿ™๐Ÿฃ๐Ÿญ๐ŸทI๏ผฉโ… โ„โ„‘๐ˆ๐ผ๐‘ฐ๐“˜๐•€๐•ด๐–จ๐—œ๐˜๐™„๐™ธฦ–๏ฝŒโ…ผโ„“๐ฅ๐‘™๐’๐“๐“ต๐”ฉ๐•๐–‘๐—…๐—น๐˜ญ๐™ก๐š•ว€ฮ™๐šฐ๐›ช๐œค๐ž๐ž˜โฒ’ะ†ำ€โตแ›๊“ฒ๐–ผจ๐ŠŠ๐Œ‰][pโด๏ฝ๐ฉ๐‘๐’‘๐“…๐“น๐”ญ๐•ก๐–•๐—‰๐—ฝ๐˜ฑ๐™ฅ๐š™ฯฯฑ๐›’๐› ๐œŒ๐œš๐†๐”๐ž€๐žŽ๐žบ๐Ÿˆโฒฃั€]

instead of this, which is valid and desired:

[h๏ฝˆโ„Ž๐ก๐’‰๐’ฝ๐“ฑ๐”ฅ๐•™๐–๐—๐—ต๐˜ฉ๐™๐š‘าปีฐแ‚]**[**eโ„ฎ๏ฝ…โ„ฏโ…‡๐ž๐‘’๐’†๐“ฎ๐”ข๐•–๐–Š๐–พ๐—ฒ๐˜ฆ๐™š๐šŽ๊ฌฒะตาฝaโบ๏ฝ๐š๐‘Ž๐’‚๐’ถ๐“ช๐”ž๐•’๐–†๐–บ๐—ฎ๐˜ข๐™–๐šŠษ‘ฮฑ๐›‚๐›ผ๐œถ๐ฐ๐žชะฐ**]**[l๐Ÿ๐Ÿ™๐Ÿฃ๐Ÿญ๐ŸทI๏ผฉโ… โ„โ„‘๐ˆ๐ผ๐‘ฐ๐“˜๐•€๐•ด๐–จ๐—œ๐˜๐™„๐™ธฦ–๏ฝŒโ…ผโ„“๐ฅ๐‘™๐’๐“๐“ต๐”ฉ๐•๐–‘๐—…๐—น๐˜ญ๐™ก๐š•ว€ฮ™๐šฐ๐›ช๐œค๐ž๐ž˜โฒ’ะ†ำ€โตแ›๊“ฒ๐–ผจ๐ŠŠ๐Œ‰][pโด๏ฝ๐ฉ๐‘๐’‘๐“…๐“น๐”ญ๐•ก๐–•๐—‰๐—ฝ๐˜ฑ๐™ฅ๐š™ฯฯฑ๐›’๐› ๐œŒ๐œš๐†๐”๐ž€๐žŽ๐žบ๐Ÿˆโฒฃั€]

(** only added for emphasis)

JoshuaS3 avatar Mar 17 '23 19:03 JoshuaS3