go-astisub icon indicating copy to clipboard operation
go-astisub copied to clipboard

Implement custom `bufio.SplitFunc` for WebVTT

Open nakkamarra opened this issue 4 months ago • 1 comments

From an excerpt of the WebVTT spec regarding the WebVTT parsing algorithm:

Let input be the string being parsed, after conversion to Unicode, and with the following transformations applied: Replace all U+0000 NULL characters by U+FFFD REPLACEMENT CHARACTERs.

Replace each U+000D CARRIAGE RETURN U+000A LINE FEED (CRLF) character pair by a single U+000A LINE FEED (LF) character.

Replace all remaining U+000D CARRIAGE RETURN characters by U+000A LINE FEED (LF) characters.

I don't believe the current astisub WebVTT parsing logic is covering this part of the WebVTT parsing algorithm correctly. The bufio.Scanner is using the default which is ScanLines and I'm not sure this is handling the WebVTT parsing correctly (i.e a file with exclusively \r terminations will not be replaced and thus not parsed properly). So I'm thinking maybe implementing a new bufio.SplitFunc and feeding it to the scanner.Split(...)

This somewhat ties into another issue I am currently trying to make a PR to solve #110, where I'd like to:

  1. read the first 6 bytes and make sure they are WEBVTT
  2. read the remainder of the content up until the first occurrence of two successive line terminations
  3. write that content into a new field of the Subtitles struct that has each line

Thoughts? Maybe there's a better way for me to achieve what I'm trying to do.

nakkamarra avatar Oct 09 '24 03:10 nakkamarra