CSwiftV
CSwiftV copied to clipboard
wrong parsing
"They used to work here, _____ they?",hadn't,weren't,didn't,used not,0,0,1,0,4
Created through Microsoft Excel, but "They used to work here, _____ they?" parsed as two different tokens, instead of one.
So quotes are ignored, it seems...
Thanks for the issue. For reference - what is the your expected outcome for the above string?
I assume:
[
"They used to work here, _____ they?",
"hadn't",
"weren't",
"didn't",
"used not",
"0",
"0",
"1",
"0",
"4"
]
Is that correct?
@Daniel1of1 yes, so if the string starts with "
then the whole token is up to ",
, as far as I understand from excel.
Also there could be "
inside token, they should be left as is.
Incase this is still an issue for you, the latest develop
should have this behaviour now.
And to make clear my previous point expected string should actually be: (different first line)
[
"\"They used to work here, _____ they?\"",
"hadn't",
"weren't",
"didn't",
"used not",
"0",
"0",
"1",
"0",
"4"
]
Right?
( no worries if this long past being an issue 😊 )