spotify-downloader
spotify-downloader copied to clipboard
spotDL
OSError: Too Many Files, when downloading a bigger playlist
System OS
MacOS
Python Version
3.11 (CPython)
Install Source
pip / PyPi
Install version / commit hash
v4.2.4
Expected Behavior vs Actual Behavior
When downloading a larger playlist after about 50-70 tracks have been downloaded the terminal throws an error saying Too many open files, and all subsequent tracks do not get downloaded and it starts saying audio provider error.
Steps to reproduce - Ensure to include actual links!
Simple spotdl command on any large playlist should show this issues
Traceback
OSError: [Errno 24] Too many open files
OSError: [Errno 24] Too many open files
AudioProviderError: YT-DLP download error - https://music.youtube.com/watch?v=ZiZWzVfeWVg
AudioProviderError: YT-DLP download error - https://music.youtube.com/watch?v=Aduc9vE3Mog
AudioProviderError: YT-DLP download error - https://music.youtube.com/watch?v=_GfaWmDDpNM
AudioProviderError: YT-DLP download error - https://music.youtube.com/watch?v=XHA8OP2IbXU
Other details
No response
gretting, since you're using mac, please tell me the value of "open file" using ulimit -a
default number of open file descriptors on mac is 256. This behavior does not goes away when using the --threads flag
❯ ulimit -n
256
this tutorial solved temporarily the issue: https://gist.github.com/skylock/0117ec637d468f91260927b43b816eda
but closing the file after saving it should solve this issue (file stay open in RAM even after song finished)
looking through the source quickly it looks like the project follows the pythonic with in a few different places but doesn't explicitly call os.close which I think is the source of the issue
ref: https://github.com/spotDL/spotify-downloader/blob/f90cfb3f5ab1b15e73d0393d531908d4ac789e7d/spotdl/console/save.py#L87-88
Confirming this issue on Linux (using Ubuntu 23). This starts to happen after downloading a couple hundred files. Max open files on my OS is not altered, so should be whatever the default is.
Downloaded "Le Roy - See the Light Ahead": https://music.youtube.com/watch?v=Kd5reUZaGEI OSError: [Errno 24] Too many open files AudioProviderError: YT-DLP download error - https://music.youtube.com/watch?v=aEV83SlKyU0 OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt'
I'm having the same issue when downloading a lot of files from WSL using Ubuntu.
For reference, I'm using the command: spotdl download saved --user-auth --lyrics musixmatch --output "{album-artist}/{album-artist} - {title}"
Downloaded "ILLENIUM - Where'd U Go (Thoreau Rework)": https://music.youtube.com/watch?v=ZlTI54Bwe1E
OSError: [Errno 24] Too many open files
Downloaded "The Him - Broken Love (feat. Parson James)": https://music.youtube.com/watch?v=UMygH95XZqY
OSError: [Errno 24] Too many open files
AudioProviderError: YT-DLP download error - https://music.youtube.com/watch?v=lRJnVT_OPhk
Couldn't read cache at: /home/antoinedelia/.spotdl/.spotipy
OSError: [Errno 24] Too many open files
Couldn't read cache at: /home/antoinedelia/.spotdl/.spotipy
OSError: [Errno 24] Too many open files
Couldn't read cache at: /home/antoinedelia/.spotdl/.spotipy
OSError: [Errno 24] Too many open files
Couldn't read cache at: /home/antoinedelia/.spotdl/.spotipy
OSError: [Errno 24] Too many open files
Couldn't read cache at: /home/antoinedelia/.spotdl/.spotipy
OSError: [Errno 24] Too many open files
When I'm running ulimit -n, I got 1024.
try running ulimit -S 4096
I will investigate this more thoroughly whenever I get some time
I was able to fix it by using the docker container, since I believe the Docker VM is setup with a much higher "open file" limit. I tested it on a 1300 song playlist and it was able to download ~400 songs before erroring out. The code definitely still needs a fix to close files after it is done with with them, but as a temporary workaround you can use the docker container and run it a few times with the "sync" command.
docker run --rm -v $(pwd):/music spotdl/spotify-downloader sync https://open.spotify.com/playlist/playlistURL --save-file playlist.spotdl --format mp3 --threads 8
I am experiencing this problem on Ubuntu.
OSError: [Errno 24] Too many open files
As someone else suggested, I'm thinking the file handles are not being closed. If the code is recursive or the file handle isn't explicitly closed then it could cause this error. I have yet to look at the source code.
I wrote a little bash script to kill and restart every 30 mins with a crontab.
