go-astisub
go-astisub copied to clipboard
Implement custom `bufio.SplitFunc` for WebVTT
From an excerpt of the WebVTT spec regarding the WebVTT parsing algorithm:
Let input be the string being parsed, after conversion to Unicode, and with the following transformations applied: Replace all U+0000 NULL characters by U+FFFD REPLACEMENT CHARACTERs.
Replace each U+000D CARRIAGE RETURN U+000A LINE FEED (CRLF) character pair by a single U+000A LINE FEED (LF) character.
Replace all remaining U+000D CARRIAGE RETURN characters by U+000A LINE FEED (LF) characters.
I don't believe the current astisub WebVTT parsing logic is covering this part of the WebVTT parsing algorithm correctly. The bufio.Scanner
is using the default which is ScanLines and I'm not sure this is handling the WebVTT parsing correctly (i.e a file with exclusively \r
terminations will not be replaced and thus not parsed properly). So I'm thinking maybe implementing a new bufio.SplitFunc
and feeding it to the scanner.Split(...)
This somewhat ties into another issue I am currently trying to make a PR to solve #110, where I'd like to:
- read the first 6 bytes and make sure they are
WEBVTT
- read the remainder of the content up until the first occurrence of two successive line terminations
- write that content into a new field of the Subtitles struct that has each line
Thoughts? Maybe there's a better way for me to achieve what I'm trying to do.