docker-handbrake
docker-handbrake copied to clipboard
jlesage
Help needed getting hardware acceleration started
Sorry for asking. I'm new in Synology Docker experience. So likely it's an absolute silly question.
First of all: The experience with your HandBrake solution is great so far (after an longer time with problems caused only by myself until I got the conclusion between your written down start command and the transfer to the synology docker GUI). Thank you so much!
So I wanted to do the next step now. I own a DS218+ with Intel processor listed for hardware acceleration.
My stupid question is: where do I have to write down the command "--device /dev/dri" in the Docker GUI?
As there was no clue for me, I tried this command via ssh form my pc:
sudo docker run -d --name=handbrake --device /dev/dri -e USER_ID=1026 -e-GROUP_ID=1000 -p 5803:5800 -v /volume1/homes/christian/HandBrake/config:/config:rw -v /volume1/homes/christian/HandBrake/storage:/storage:ro -v /volume1/homes/christian/HandBrake/watch:/watch:rw -v /volume1/homes/christian/HandBrake/output:/output:rw jlesage/handbrake
Took me a longer time to realize that I have to make sudo to make it. But after all a new container was started. Poorly when I open the HandBrake-GUI nevertheless there is no "H.264 (Intel QSV)".
Your advice is to check the container's log for the issue. Unfortunately due to lack of knowledge I'm not able to see what's the reason.
Could you give me a hand?
Mevertheless my prefered way would be using the GUI because I'm not that firm with command line.
handbrake_Prozesslog_Hardwarebeschleunigung.html.pdf
Hello, maybe i can help you or give you some information at least. Btw. i am struggeling with similar problems. I have hardware acceleration running with almost the same configuration. My Syno is a 718+ modell.
Look at my questions here: https://github.com/jlesage/docker-handbrake/issues/32
This is my command to run the container from a planed Synology Task: #!/bin/bash sudo docker run --name handbrake2 --net=host --restart always --volume /volume1/docker/handbrake/config:/config:rw --volume /volume1/NeueFilme:/output:rw --volume /volume1:/storage:ro --publish 5800:5800 --env USER_ID=0 --env GROUP_ID=101 --device /dev/dri jlesage/handbrake:dev-latest
I am sorry, for my knowledge there is no way to pass the "--device /dev/dri" command in the Synology GUI, thats why i use the command - but got problems with that too.
I am from germany too, active in the kodinerds forum: https://www.kodinerds.net/index.php/User/35876-Commerzpunk/
Maybe you wanna go there to discuss in german? There are a few guys with knowledge in docker and some with handbrake too.
Could you also provide the following log file:
/volume1/homes/christian/HandBrake/config/ghb/Activity.log.XXXX.
Can you try to re-create the container, but instead of using USER_ID=1026, try with USER_ID=0.
Thank you very much. Works for me to get option "H.264 (Intel QSV)".
But the same issue occurs that RealCommerzpunk described: I have not even one video in the watch folder actual but the cpu is running like on the run.
.
So you only changed the value of USER_ID and see the CPU usage bump?
Can you try to re-create the container, but this time without the --device /dev/dri parameter?
I did as you told me
sudo docker run -d --name=handbrake2 -e USER_ID=0 -e-GROUP_ID=1000 -p 5803:5800 -v /volume1/homes/christian/HandBrake/config:/config:rw -v /volume1/homes/christian/HandBrake/storage:/storage:ro -v /volume1/homes/christian/HandBrake/watch:/watch:rw -v /volume1/homes/christian/HandBrake/output:/output:rw jlesage/handbrake
and - wow - CPU is up to ~38%
So the same behavior as with "device"
Ok so the issue seems to be related to the fact that the container is running as root.
Alright, lets go on here because i think this issue is somewhere arround "user" and "hardware acceleration". My issue still isnt solved.
I tried a lot last night, put literally everything upside down and got a whole new setup.
- Stopped using a command to create the container, only the Synology UI
- Changed the rights to the "dev/dri/*" path to 777
- Inserted a startup script to set those rights on every start as Synology seems to reset them
- So there is no longer need to pass the "device"
- Did and did not use the privileged mode ("--privileged")
- Played with UID und GID from 0 (root) to 1026 (my very own user) and 1024 (the admin) and the corresponding groups
What i did NOT:
- Use the bridge mode
- Use the latest tag image (but i think i sorted out that is not the solution).
What ever i tried
- either got no hardware acceleration availible
- or got permission errors for my shares
- or got the massive high CPU usage and "only one file in a row" issue
Its not working all together, one issue is always left.
Finally i am at the very first point again, acceleration is working, i can transcode - but only one file and a high CPU usage. So in the end: A whole new setup and the same issues.
Ideas?
Ok so the issue seems to be related to the fact that the container is running as root.
No, its not. My container runs with "1024" and "101" which is my admin and the administrators group: And its not in "privileged" mode.
admin@nas-27:~$ id uid=1024(admin) gid=100(users) groups=100(users),101(administrators)
I am pretty sure yes, but can tell you for sure in about an hour when my latest recording is transcoded. ;-)
I found an issue with the auto video converter where a loop is created when running as root. This certainly explain the high cpu usage. I'm working on a fix.
I am very glad to hear this! Good luck.
Here is my update: My container runs with IUD "1024" and GID "101" which is my admin and the administrators group: And it was "privileged" mode. (which is the same as "run as root" as i understand it).
When i run it not in privileged mode, i dont have the QSV options, so HW acceleration is not availible.
Lets see what you come up with!
And it was "privileged" mode. (which is the same as "run as root" as i understand it).
Running as root and privileged mode are 2 different things. Running as root means that the processes in the container run as user ID 0. This doesn't affect privileges of the container itself nor its capabilities.
Activating the privileged mode gives all capabilities to the container, provide full access to all devices on the host, etc.
Thank you very much. I'll test it in the evening when back from work. I will report.
Updated. Looking very good so far. After start CPU is rising up to 17 % for a moment and immediately going down (no video in watch). And...."H.264 (Intel QSV)" is selectable in HandBrake.
And further on. Finaly now on 0,6 % on idle.
Will try transcoding now.
I did the update too > latest running in bridged mode,priviledged, UID 0
good:
- QSV is availible
- CPU usage goes down after use
- CPU usage low in idle (after starting the container)
not good:
- still can only encode 1 file in a row, not a queue (fails after first transcode as before)
- CPU usage goes down in very very small steps, like from 9% to 3,4% atm in 15 minutes. (UPDATE, it went down to 0,52 % after more then an hour)
See these 2 logs, its the exact same file (but copied, so 2 seperated input files). Starting a queue with the exact same settings and fired. No. 1 did work and created a output, No. 2 did not even create a output video file. The last line is "libhb: work result = 3" Googling a little bit about it showed some information about permission issues. Maybe thats a hint?
1SampleVideo_1280x720_20mb 2018-11-13 17-20-04.log
2SampleVideo_1280x720_20mb 2018-11-13 17-21-12.log
Mmm. Seems I do something wrong. Nothing happens.
This is the way I did:
Started Container.
Created a new preset and set it on default in the HandBrake GUI.
Stopped the container.
Wrote the name of the new preset into Automated_Conversion.
Started the container.
Nothing happens except for a moment CPU goes up i red zone and than down to I guess idle.
Activity.log.982.pdf
handbrake_Acceleration.html.pdf
So got it running at least. Took the Preset in the terminal:
sudo docker run -d --name=handbrake1 --device /dev/dri -e USER_ID=0 -e GROUP_ID=1000 -e AUTOMATED_CONVERSION_PRESET=Hardware_Acceleration -p 5800:5800 -v /volume1/homes/christian/HandBrake/config:/config:rw -v /volume1/homes/christian/HandBrake/storage:/storage:ro -v /volume1/homes/christian/HandBrake/watch:/watch:rw -v /volume1/homes/christian/HandBrake/output:/output:rw jlesage/handbrake
Running now. Curious to see the result tomorrow. GN8@all
@RealCommerzpunk, I think the issue you have about not being able to encode 2 files is related to the QSV H265 encoder. Do you have the same problem with the H264 version? Also, do you have the issue with the automatic video converter? For the CPU issue, can you check which process is taking the CPU? By the way, there is no need to run the container with privileged mode.
Starting and running good so far. 👍(Had no queue like RealCommerzpunk at the moment. So I can't report to that issue). Next will be to make some comparison of different presets in Handbrake and their effects on quality, duration and workload.
Thank you very much for your dedicated work! Is there a piggybank to reveal?
@jlesage You are right. Using the WebGUI it only fails to encode more then one file in a queue when using H265 QSV. H264 QSV works and few others i tried.
But now i am stunned. Using the watch folders, its even processing more then one file! Using the same preset which uses h265 qsv of course!
One more thing: When ever a conversion fails this way, the h265 qsv codec seems to be blocked. Only after restarting the container its possible to encode one file again. This happens to the watch folders too. So if the codec is blocked, even watch folders dont work, failing too. After a fresh start of the container several files can be transcoded via watch folders.
Does that help you to catch this nasty bug? Anything i can help? Logs, testing scenarios?
Can you try again, but by adding the HANDBRAKE_DEBUG=1 environment variable when creating the container? Reproduce the issue and please provide log/hb/handbrake.debug.log and ghb/Activity.log.XXX. These files are in the folder you mapped to /config.
Sorry, that took a while. @jlesage You told me there is no need to run the container with privileged mode. For me there is a Need, because if i dont, there is no QSV Option availible.
Today i finally did the logs you asked for, see attached.
handbrake.debug.log Activity.log queue.log
Activity.log and Queue.log have be renamed be me to upload them. Before the Suffix was "991".
Here is a screen what it Looks like after finishing a Job and using too much CPU:
You told me there is no need to run the container with privileged mode. For me there is a Need, because if i dont, there is no QSV Option availible.
Then there is probably a permission issue on the host with the device. Make sure you have the latest image and look at the container log during the startup: it should indicate why QSV can't be used.
For the QSV H265 issue, I suspect it's an issue/bug with HandBrake (i.e. not sure it's an issue with the container). I can try to report the issue to the HandBrake team and see what they think about this. I would use your log if you don't mind, since I can't reproduce the issue (I don't have a CPU supporting H265 QSV).
Also, could you confirm that QSV H265 issue is reproducible with the latest nightly/development build (v1.14.9-dev-20181124-2b2bf33f47)?
@RealCommerzpunk, can you try the tag v1.14.10-beta1 to see if it fixes the H265 QSV issue?
Hello, i made a all fresh container with the v1.14.10-beta1 image. All new even a new empty config folder. Running as UID 0, priviledged mode, bridge network.
Took the default preset, just changed it to MKV and H265 QSV, grabbed my 2 testfiles, put them in a queue.
Same issue as before, first one succeeds, second fails. Activity.log.txt queue.txt
Logs attached.
Thanks for the report. Note that I also pushed the v1.14.10-beta3 image. This version should also fail, as confirmed by another user, but if you can also confirm this it would be great.
Yes, it its exactly the same with beta3 imaga. Same procedure as before.
Hey! That did it! Beta 5 did run my 2 testfiles from the Queue in the row! First time since month! Cool!
Any logs you Need?
Thanks for the verification! Logs not needed. An official build will be available today :)
For those not wanting to use the docker CLI, I got this working following this guide: https://wildestpixel.co.uk/enabling-quick-sync-video-on-handbrake-for-docker-on-synology-2
I too am a big fan of the container and thank you @jlesage .
@muellerk22 Could you repost the link? I did manage to unlock the h.264 QSV and h.265 QSV encoding, however only h.264 works and the other crashes at the beginning...
