TokensRegex cannot detect rules cross special symbols, eg. '.' or ','

[lilyclemson]

• I have a test string "I live in 123 Pretty RD, APT. # 456." and expected to parse the apartment address out using below rules:

{ ruleType: "tokens", pattern: ( /[0-9]+/ /([A-Z][A-Za-z]+[ ,]?)/+ /[Ave|St|Blvd|Apt|APT|RD]+[,.]?/ /# [0-9]+/), action: Annotate($0, ner, "APT"), result:"APARTMENT"}

• However, it looks like TokensRegex cannot detect rules cross special symbols, eg. '.' or ','

• The original issue was #1396. It was closed by a fix in V4.5.6. It seems like I cannot reopen the issue. Thus create a new issue for it. Please let me know if there is a way to reopen it.

Please help take a look into this issue. Thank you!