File Access Error

[pistolwhip-dev]

Not 100% on the cause, but ~~I suspect PUID PGID is~~ something messing with the ability for yt-dlp to access the cache directory for part downloads, resulting in failure to get videos:

  File "/usr/local/lib/python3.9/dist-packages/yt_dlp/cache.py", line 49, in store
    os.makedirs(os.path.dirname(fn))
  File "/usr/lib/python3.9/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.9/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.9/os.py", line 225, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/root/.cache'

EDIT: tested a completely default run with pre-applied correct 911:911 ownership for the two required mounts, same result. config access is fine, dl directory seems fine testing from in the container, but the /root/.cache dir isn't open for yt-dlp to use for processing.

[Jeeaaasus]

Hello @pistolwhip-dev 🙂

interesting issue, I am unable to replicate it. A couple of question: Are you creating the container as root (using sudo or equivalent)? Is this with the image named latest? Is the problem present with images jeeaaasustest/youtube-dl:v2021.12.25 or jeeaaasustest/youtube-dl:v2021.12.01?

I have some ideas but need more information, if you could please post the full logs of the issue happening after creating the container with env -e youtubedl_debug=true

Appreciate your help, thank you!

[pistolwhip-dev]

Sure, some additional details: Debian 10.11 kernel 4.19.0-18-amd64 x86_64 Docker CE (from docker repos) 20.10.9, build c2ea9bc

I do not run docker as root, instead make use of the Docker user group (which is functionally the same thing as I understand it)

Here is my compose:

services:
  yt-dlp:
    environment:
      - PUID=1001
      - PGID=1001
      - youtubedl_debug=true
    image: jeeaaasustest/youtube-dl:latest
    container_name: yt-dlp
    volumes:
      - ./data:/config
      - /opt/tank/media/yt:/downloads
    restart: unless-stopped
    logging:
      driver: "json-file"
      options:
        max-size: "200k"
        max-file: "10"

Results from specified image tags: v2021.12.25

New error in init regarding updater.sh

yt-dlp    | chmod: cannot access '/app/youtube-dl/updater.sh': No such file or directory

Same exact error post-init with permission denied.

v2021.12.01

Seems to behave the same as 12.25

the following is log output for a fresh run of latest with the aforementioned compose

Attaching to yt-dlp
yt-dlp    | [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
yt-dlp    | [s6-init] ensuring user provided files have correct perms...exited 0.
yt-dlp    | [fix-attrs.d] applying ownership & permissions fixes...
yt-dlp    | [fix-attrs.d] done.
yt-dlp    | [cont-init.d] executing container initialization scripts...
yt-dlp    | [cont-init.d] 05-default-confs: executing...
yt-dlp    | [cont-init.d] 05-default-confs: exited 0.
yt-dlp    | [cont-init.d] 10-update-args: executing...
yt-dlp    | [cont-init.d] 10-update-args: exited 0.
yt-dlp    | [cont-init.d] 80-webui: executing...
yt-dlp    | [web-ui] webserver is disabled.
yt-dlp    | [cont-init.d] 80-webui: exited 0.
yt-dlp    | [cont-init.d] 90-user-permissions: executing...
yt-dlp    | [cont-init.d] 90-user-permissions: exited 0.
yt-dlp    | [cont-init.d] 99-init.d-finish: executing...
yt-dlp    |
yt-dlp    | UID/GID
yt-dlp    | ---------------------------------------
yt-dlp    | User uid:  1001
yt-dlp    | User gid:  1001
yt-dlp    | ---------------------------------------
yt-dlp    |
yt-dlp    | Image Made By: Jeeaaasus
yt-dlp    | https://github.com/Jeeaaasus/youtube-dl
yt-dlp    | https://hub.docker.com/r/jeeaaasustest/youtube-dl
yt-dlp    |
yt-dlp    | [cont-init.d] 99-init.d-finish: exited 0.
yt-dlp    | [cont-init.d] done.
yt-dlp    | [services.d] starting services
yt-dlp    | [services.d] done.
yt-dlp    |
yt-dlp    | 2021-12-26 07:23:35 - starting execution
yt-dlp    | [debug] Command-line config: ['--config-location', '/config/args.conf', '--batch-file', '/app/urls', '--verbose', '--format', '((bv*[fps>30]/bv*)[height<=1080]/(wv*[fps>30]/wv*)) + ba / (b[fps>30]/b)[height<=1080]/(w[fps>30]/w)', '--download-archive', '/config/archive.txt']
yt-dlp    | [debug] Custom config file: /config/args.conf
yt-dlp    | [debug] Custom config: ['--sponsorblock-remove', 'default']
yt-dlp    | [debug] Batch file urls: ['https://www.youtube.com/channel/UCsXVk37bltHxD1rDPwtNM8Q']
yt-dlp    | [debug] Encodings: locale UTF-8, fs utf-8, out utf-8 (No ANSI), err utf-8 (No ANSI), pref UTF-8
yt-dlp    | [debug] yt-dlp version 2021.12.25 [87e049962]
yt-dlp    | [debug] Python version 3.9.2 (CPython 64bit) - Linux-4.19.0-18-amd64-x86_64-with-glibc2.31
yt-dlp    | [debug] exe versions: ffmpeg N-105024-ga1c4929f65-20211224 (setts), ffprobe N-105024-ga1c4929f65-20211224
yt-dlp    | [debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
yt-dlp    | [debug] Proxy map: {}
yt-dlp    | [debug] Loading archive file '/config/archive.txt'
yt-dlp    | [debug] [youtube:tab] Extracting URL: https://www.youtube.com/channel/UCsXVk37bltHxD1rDPwtNM8Q
yt-dlp    | [youtube:tab] UCsXVk37bltHxD1rDPwtNM8Q: Downloading webpage
yt-dlp    | WARNING: [youtube:tab] A channel/user page was given. All the channel's videos will be downloaded. To download only the videos in the home page, add a "/featured" to the URL
yt-dlp    | [debug] [youtube:tab] Final URL: https://www.youtube.com/channel/UCsXVk37bltHxD1rDPwtNM8Q/videos
yt-dlp    | [download] Downloading playlist: Kurzgesagt – In a Nutshell - Videos
yt-dlp    | [youtube:tab] UCsXVk37bltHxD1rDPwtNM8Q page 1: Downloading API JSON
yt-dlp    | [youtube:tab] UCsXVk37bltHxD1rDPwtNM8Q page 2: Downloading API JSON
yt-dlp    | [youtube:tab] UCsXVk37bltHxD1rDPwtNM8Q page 3: Downloading API JSON
yt-dlp    | [youtube:tab] UCsXVk37bltHxD1rDPwtNM8Q page 4: Downloading API JSON
yt-dlp    | [youtube:tab] UCsXVk37bltHxD1rDPwtNM8Q page 5: Downloading API JSON
yt-dlp    | [youtube:tab] playlist Kurzgesagt – In a Nutshell - Videos: Downloading 153 videos
yt-dlp    | [download] Downloading video 1 of 153
yt-dlp    | [youtube] xAUJYP8tnRE: Downloading webpage
yt-dlp    | [debug] [youtube] Extracting URL: https://www.youtube.com/watch?v=xAUJYP8tnRE
yt-dlp    | [youtube] xAUJYP8tnRE: Downloading android player API JSON
yt-dlp    | [youtube] xAUJYP8tnRE: Downloading player 8da38e9a
yt-dlp    | WARNING: Writing cache to '/root/.cache/yt-dlp/youtube-nsig/8da38e9a.json' failed: Traceback (most recent call last):
yt-dlp    |   File "/usr/local/lib/python3.9/dist-packages/yt_dlp/cache.py", line 49, in store
yt-dlp    |     os.makedirs(os.path.dirname(fn))
yt-dlp    |   File "/usr/lib/python3.9/os.py", line 215, in makedirs
yt-dlp    |     makedirs(head, exist_ok=exist_ok)
yt-dlp    |   File "/usr/lib/python3.9/os.py", line 215, in makedirs
yt-dlp    |     makedirs(head, exist_ok=exist_ok)
yt-dlp    |   File "/usr/lib/python3.9/os.py", line 225, in makedirs
yt-dlp    |     mkdir(name, mode)
yt-dlp    | PermissionError: [Errno 13] Permission denied: '/root/.cache'
yt-dlp    |
yt-dlp    | [debug] [youtube] Decrypted nsig uCv4WFdncP-MnHzi => r7ogoGc8okX7_A
yt-dlp    | [debug] Sort order given by extractor: quality, res, fps, hdr:12, source, codec:vp9.2, lang, proto
yt-dlp    | [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
yt-dlp    | [SponsorBlock] Fetching SponsorBlock segments
yt-dlp    | [debug] SponsorBlock query: https://sponsor.ajay.app/api/skipSegments/8bce?service=YouTube&categories=%5B%22outro%22%2C+%22music_offtopic%22%2C+%22filler%22%2C+%22sponsor%22%2C+%22selfpromo%22%2C+%22intro%22%2C+%22preview%22%2C+%22interaction%22%5D
yt-dlp    | [SponsorBlock] Found 3 segments in the SponsorBlock database
yt-dlp    | [info] xAUJYP8tnRE: Downloading 1 format(s): 303+251
yt-dlp    | [debug] Invoking downloader on "https://rr1---sn-ab5szn76.googlevideo.com/videoplayback?expire=1640525019&ei=exjIYanMKdGO8wSS8pCoCg&ip=50.220.58.158&id=o-AKoYoX24XCbVxlJcpAiCsIHgbI2WT3czxwy-mv0Mw5vM&itag=303&source=youtube&requiressl=yes&mh=qU&mm=31%2C26&mn=sn-ab5szn76%2Csn-vgqsknly&ms=au%2Conr&mv=u&mvi=1&pl=23&vprv=1&mime=video%2Fwebm&gir=yes&clen=127060169&dur=731.516&lmt=1639751061194051&mt=1640502773&fvip=1&keepalive=yes&fexp=24001373%2C24007246&c=ANDROID&txp=5535434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRAIgRJyvd-9bw90hmMU6VdM2TLzgD3LwyFeLSJzwAu4lKLsCICjXf11-2tNZiu3oy6VjCDKHDevjta-wytoGVOK4x5wL&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl&lsig=AG3C_xAwRQIgCQq5Qma_FAqNEYHAlzm4hkFDPEaAiKZpuWmiKfd0l2ECIQDvR026E3O9fSzwn8o6aNw0vZL5z-BIYx6nkos_rweLPQ%3D%3D"
yt-dlp    | [download] Got file access error. Retrying (attempt 1 of 10) ...
yt-dlp    | [download] Got file access error. Retrying (attempt 2 of 10) ...
yt-dlp    | [download] Got file access error. Retrying (attempt 3 of 10) ...
yt-dlp    | [download] Got file access error. Retrying (attempt 4 of 10) ...
yt-dlp    | [download] Got file access error. Retrying (attempt 5 of 10) ...
yt-dlp    | [download] Got file access error. Retrying (attempt 6 of 10) ...
yt-dlp    | [download] Got file access error. Retrying (attempt 7 of 10) ...
yt-dlp    | [download] Got file access error. Retrying (attempt 8 of 10) ...
yt-dlp    | [download] Got file access error. Retrying (attempt 9 of 10) ...
yt-dlp    | [download] Got file access error. Retrying (attempt 10 of 10) ...
yt-dlp    | ERROR: unable to open for writing: [Errno 13] Permission denied: 'Why We Should NOT Look For Aliens - The Dark Forest [xAUJYP8tnRE].f303.webm.part'
yt-dlp    | Traceback (most recent call last):
yt-dlp    |   File "/usr/local/lib/python3.9/dist-packages/yt_dlp/downloader/http.py", line 265, in download
yt-dlp    |     ctx.stream, ctx.tmpfilename = self.sanitize_open(
yt-dlp    |   File "/usr/local/lib/python3.9/dist-packages/yt_dlp/downloader/common.py", line 218, in sanitize_open
yt-dlp    |     return sanitize_open(filename, open_mode)
yt-dlp    |   File "/usr/local/lib/python3.9/dist-packages/yt_dlp/utils.py", line 557, in sanitize_open
yt-dlp    |     stream = open(encodeFilename(filename), open_mode)
yt-dlp    | PermissionError: [Errno 13] Permission denied: 'Why We Should NOT Look For Aliens - The Dark Forest [xAUJYP8tnRE].f303.webm.part'

[Hamitamaru]

I was hitting similar issues where it could not write files inside the container's filesystem so I passed the following options to my yt-dlp.conf

--dateafter now-2days
--cache-dir /downloads/cache
--verbose
-P "temp:/temp"
-P "home:/downloads"

needed the temp path override here and the /temp folder in the container is mounted to the folder on the host system ( e.g. /youtube-dl/temp on host ) otherwise it always failed to downloaded the pieces of the video to the temp directory

needed the home path override here and the /downloads folder in the container is mounted to the folder on the host system ( e.g. /youtube-dl/downloads on host ) otherwise ffmpeg always failed to merge the downloaded pieces into a single file (at least I assume that was what it was trying to do)

hope this gets you past your issue as well, though I'm not certain if this is the correct path forward my fresh install is now working as intended.

[pistolwhip-dev]

Can confirm, the work-around suggested by @Hamitamaru do let the container operate as expected.

[Jeeaaasus]

I'm gonna close this issue because the workaround provided by @Hamitamaru, thank you!, seems to work for @pistolwhip-dev.

BUT I am willing to try to fix this issue for real, IF anyone with this issue wants it fixed and wants to help me fix it, just open a new issue and we can try. 😊