[Bug] Random failures with buffered input on stdin with certain files

[mackworth]

Per @cfsmp3 request on Issue #436, I am opening a new issue on this bug. I have confirmed it is still a problem.

Just updated my copy to the latest master from github, recompiled on my Mac and have the exact same symptoms.

To be specific, when run through stdin pipe, the sample file available here, has a high probability of failing (averaging 7 successes out of 20) with Error: Syntax problem: Final 0xFF marker missing. The log file says TS continuity counter not incremented prev/curr 4/15. But depending on the run, I've seen 2, 9, or 15 for the "curr" number.

To recreate in Bash, set f to test file and ccextractor to compiled exe, then run for i in $(seq 1 1 20); do cat "$f" | $ccextractor -bi - -o junk3.srt > "junkNew$i.log"; if [ $? -eq 0 ]; then echo "OK "; else echo "Fail"; fi; done

CCExtractor version : 0.87

In raising this issue, I confirm the following (please check boxes, eg [X] - and delete unchecked ones):

• [X ] I have read and understood the contributors guide.

• [X ] I have checked that the bug I am reporting can be replicated

• [ X] I have checked the pull requests tab for existing solutions/implementations to my issue/suggestion.

• [X ] I have used the latest available version of CCExtractor to verify this issue exists.

• [X ] I absolutely love CCExtractor, but have not contributed previously.

Necessary information

• Is this a regression (did it work before)? - Don't know.
• What platform did you use? - [X ] Mac
• What were the used arguments? ccextractor -bi - -o junk3.srt > "junkNew$i.log";

**Video links (replace text below with your links) **

https://www.dropbox.com/s/zo6ta9d8ppl093p/RobotChicken-RandomFailuresBI.mpg?dl=0

[mackworth]

So the good news is this looks like this bug was patched over in 0.88. (From Release notes: - Fix: Warn instead of fatal when a 0xFF marker is missing' )

However, the log still reports random failures (11/20 on my last run). It still says TS continuity counter not incremented prev/curr 4/15.(and depending on the run, I've seen 2, 9, or 15 for the "curr" number, but always 4 for the prev). But now v.88 follows it with Warning! Syntax problem: Final 0xFF marker missing. Continuing... and dumps a bunch of video data (7Mb log file v 7K on a 'successful' run).

So given we're using the same video over and over and getting different results, it still looks to me like some uninitialized variable affecting the curr value, which is then triggering the FF problem.

On the other hand, the .SRT file generated is now close to correct. The FF issue skips a bit of video, as comparing the results gives:

Runs with 'TS continuity' error msg: They're supposedo one the most gentle (16 seconds in; correct otherwise) Runs without error msg They're supposed to be one of the most gentle

Randomly generating a file with a minor glitch is certainly far better than the randomly failing, but I don't think the underlying bug is fixed.

Attached are the log files and SRTs for a good run and a bad run, as well as the shell script I used. Archive.zip From our previous discussion, note that this bug only happens if the video be piped in (in this case from cat).

The video is still on Dropbox. Note that I added a .jnk as there was an issue before about downloading a video file; just remove that extension.

My binary copy of ccextractor is here (compiled from source yesterday)

Let me know if there's anything else I can provide.

[cfsmp3]

@mackworth is this still happening with current master?

[mackworth]

so, the literal answer to your question is yes, I'm seeing exactly the same results as in Nov, but I'm also still seeing version 0.88 in master (with a reference in changes.txt to a version 0.89 TBD). Is there a different branch I should use?

[cfsmp3]

No, current master. We still haven't bumped the version number, even though it has a significant amount of changes.

[mackworth]

Okay, then, yes, I did a git pull and rebuilt prior to running the test again, but got almost the exact same results. Still seeing about 5-9 "good" runs out of twenty tries. The bad runs still drop exactly the same characters.

Comparing the "good runs" logs to an old good log with debug off, I see "Filter profanity: No" and dozens of "WARNING:get_decoder_line_encoded:Invalid Color index Selected 151587081" msgs (not in the variable region), but otherwise identical. Identical differences in the bad logs.

Same difference between good and bad runs. Identical until XDS Notice: Program is now Robot Chicken, Easter Basket then the good one says:

XDS: CGMS: One generation of copies can be made
XDS: APS: No APS
XDS: Redistribution Control Descriptor: 0
Streaming |  00:10
Streaming |  00:09
Streaming |  00:10
Streaming |  00:20
Streaming |  00:19
Streaming |  00:20
Streaming |  00:19
Streaming |  00:20
Streaming |  00:30
Streaming |  00:29
Streaming |  00:30
Streaming |  00:29
Streaming |  00:30
Streaming |  00:40
Streaming |  00:39

But the bad one says:

TS continuity counter not incremented prev/curr 4/2
Warning! Syntax problem: Final 0xFF marker missing. Continuing...
WARNING: Unexpected SEI unit length...trying to continue.
 Failed block (at sei_rbsp) was:
00000000 | 00 01 C0 01 01 14 04 47 B5 00 31 47 41 39 34 03  |   ¿    Gµ 1GA94 
00000016 | D4 FF FC 20 F4 FD F2 20 FF C3 23 FE 65 64 FE 03  | ‘ˇ¸ Ù˝Ú ˇ√#˛ed˛ 
00000032 | 00 FA 00 00 FA 00 00 FA 00 00 FA 00 00 FA 00 00  |  ˙  ˙  ˙  ˙  ˙  
00000048 | FA 00 00 FA 00 00 FA 00 00 FA A9 69 EB 23 BA 49  | ˙  ˙  ˙  ˙©iÎ#∫I
00000064 | 01 1A 39 D1 83 DD 25 54 ED 1A 31 74 BF 73 62 6D  |   9—É›%TÌ 1tøsbm
00000080 | 62 E5 26 7B ED 65 3F 91 27 8E 47 10 D2 10 2D B8  | bÂ&{Ìe?ë'éG “ -∏
...

[cfsmp3]

@mackworth Is the problem still happening?