phileas icon indicating copy to clipboard operation
phileas copied to clipboard

POS post filter does not handle multi-word tokens

Open jzonthemtn opened this issue 10 months ago • 0 comments

Given the input:

"George Washington was president and his ssn was 123-45-6789 and he lived at 90210."

The POS filter fails because the tokens are "George" and "Washington" individually and not "George Washington." The filter needs changed to allow for multi-word tokens.

jzonthemtn avatar Apr 03 '24 17:04 jzonthemtn