VidCoder
VidCoder copied to clipboard
RandomEngy
BUG: VidCoder stops encoding multiple files using both drag & drop and Enqueue Folder
Problem Description
I have a folder of 10 video files that I want to transcode with VidCoder.
These files scan properly one at a time, but when I tried to drag and drop all 10 at once, VidCoder processed the first 5 files and then stopped indefinitely on the 6th file.
Cancel and retry of the last 5 files gave the same result.
Likewise cancel and retry of the last 4 files.
Likewise when I canceled and tried Enqueue Folder.
Log files from those attempts are attached.
p.s. There was a long transcode running so I was not able to try restarting VidCoder
LOGS.zip
What version of VidCoder are you running?
9.18 on Windows 10
Encode Log
No response
UPDATE: After restart, with no encode running, I was able to successfully scan the 10 video files with drag & drop.
I made a fix in 10.2 Beta that resolves a problem where the batch scan would stall if there was a crash in the worker process. This might resolve your issue, or at least let the batch scan complete and report the errors at the end.
Let me know if this works for you.
Thank you. In trying to work around the problem, I found that Pausing encoding would greatly facilitate scanning of multiple files, and since then I haven't seen the problem, so it might have been a process priority issue. I am also now encoding at Low process priority, which might also be a factor. (I should have updated this Issue with that information. Apology.) My guess is that scanning is occurring at the same priority level as encoding. If so, I think it might make sense to scan at a higher process priority than encoding, which would make scanning complete more quickly. What do you think?
It might help? But it would mean I would need to break out the process priority controls to allow choosing both scan and encode priority. CPU contention shouldn't cause crashes when scanning; the code shouldn't even be aware of the encode happening, and both should be given time to execute. If there's something going on there, I'd want to fix the root issue.
I was thinking of adjusting priority algorithmically. If encoding is set for Low priority, then scan at Below Normal. If encoding is set for Below Normal, scan at Normal. Otherwise scan at Normal. Admittedly that's not perfect. But no matter. I just ran some batch scanning torture tests without pausing encoding, and the current VidCoder version did not fail. Scanning was acceptably fast. (I got occasional pausing at the 9th or 10th file in the batch, but not long enough to be significant.) So I agree the Issue is Completed and Closed. Thank you!
I spoke too soon.
After quite a few encodes, I had paused VIdCoder (9.20) and tried to scan 2 files by dragging and dropping them into Vidcoder. It scanned 1/2, but hung starting to scan 2/2. I waited a long time to be sure.
I took minidumps of all 3 processes. I can put them on Google Drive for you if that would help.
p.s. After restarting VidCoder, the 2 files scanned and queued normally.
I can look at the minidumps. If you enable Extended logging in Global options and provide the log files, that would also be helpful.
The attached minidump files are encrypted. I'll email you the password. VidCoder.zip VidCoderWorker1.zip VidCoderWorker2.zip
VidCoderWorker1: One thread is trying to start the scan. It has p-invoked into hb.dll, calling hb_scan_list, but has gotten an c0000005 error - Access violation: It tried to look into memory that wasn't mapped properly. It bubbles the error back up the CLR, but it's blocked trying to write to StdErr.
The other is a call to System.Diagnostics.Process.GetProcessesByName that the worker makes periodically to check if the UI process is still running. This one is blocked on a memory alloc, most likely caused by the first issue.
VidCoderWorker2: On one thread the scan has finished and we're calling hb_close on hb.dll. It's also throwing a c0000005 / access violation error, and also stuck trying to write the stack trace to StdErr.
Another thread is polling for scan progress, and is stuck trying to allocate memory, again most likely caused by the access violation in the first thread.
So this looks like there may be two problems:
- The original access violation in the native hb.dll
- The process is hanging instead of crashing when the error happens
Maybe number 2 has to do with how we try to capture StdErr/StdOut for reporting back to the host process. It tries to write to the console, but is blocked. I'm not sure how I can easily fix that, as non of the code in VidCoder handles that.
As for the hb.dll crash, it appears that HandBrake users have reported it as well: maybe you could look through the support threads there? Apparently there is one known issue with libdvdcss that can cause this, but you're not reading DVDs.
It's possible I could improve VidCoder's handling of this by having the UI process kill the hung worker process. Though it would be helpful to have extended logs for both the host process and the worker process to be able to see why the current code isn't doing that. The original logs you sent only included the "Scan" but if you could include all of them with extended logging enabled that would be the best.
I'm on vacation over Memorial Day. I'll try to do that as soon as I get back on Tuesday. Thank you for all your hard work! 👏
Reproduced the hang with Extended logging. Encode was in progress. I tried to scan 13 files, and it hung on file 7. I set the program to close when the encode ended so log files would be flushed and closed.
Attached ZIP of log files is encrypted with the same password.
Logs.zip
Taking a look at it now. Does this only happen when an encode is in progress? I did see some odd behavior where the scan process communication hung until the encode completed.
Thank you. See my Comment here for one case when it hung while scanning was paused. In all other cases it has hung while scanning was in progress.
The logs are definitely interesting. It's hung where I didn't expect, right when we're about to send another chunk of log to the UI process, but before it tries to take the pipe write lock.
I noticed that we are setting RedirectStandardError = true on the worker process, but we aren't using the resulting StandardError stream. I've removed RedirectStandardError in 10.6 Beta, so maybe this will resolve the odd hang on trying to write the error message to the console and let the scan process actually crash properly when HB hits the memory access violation.
From the RedirectStandardError docs:
When the child process writes enough data to fill its redirected stream, it is dependent on the parent. The child process waits for the next write operation until the parent reads from the full stream or closes the stream.
Using 10.6 Beta, I just did five batches of 15 scans while encoding at Low priority with zero hangs. 👍 Thank you!