Problem with downloading bunch of songs with Downloader

[delrius-euphoria]

I used Downloader to download a bunch of song using list of song id but the problem is that some songs don't get downloaded which causes some issues with the event loop of the app and crashes it after download but 90 + percent of songs gets download. Here is the code I use:

    playlist_tracks = deezer.get_playlist_tracks(id)
    playlist = deezer.get_playlist(id)
    playlist_name = playlist['DATA']['TITLE']

    if not os.path.exists(path+playlist_name):
        os.makedirs(path+playlist_name)
    new_path = path+playlist_name+'/'

    lst_id = [playlist_tracks[idx]['SNG_ID'] for idx,_ in enumerate(playlist_tracks)]
    # print(lst_id,len(lst_id)) 
    downloader = Downloader(deezer,lst_id,download_dir=new_path,quality=quality_format[quality_cbx.get()])
    downloader.start()

But I think it is some problem with Downloader, I could just use getting the id and getting the corresponding get_track with the id and then:

    track = deezer.get_track(id)
    track['download'](download_dir=path,quality=quality_format[quality_cbx.get()])

But when doing this for a lot of number of times raises and API error, which I lost it for now, ill update with it if I get it again.

With all this being said, I dont think this repo is getting maintained, I appreciate every help I get and the work put onto this.

EDIT: Here is the error if using track[download](...):

Traceback (most recent call last):
  File "c:\PyProjects\Music Player\lastfm.py", line 27, in <module>
    track = deezer.get_track(tracks[idx]['SNG_ID'])
  File "C:\Users\nihaa\AppData\Local\Programs\Python\Python39\lib\site-packages\pydeezer\Deezer.py", line 161, in get_track
    "tags": self.get_track_tags(data),
  File "C:\Users\nihaa\AppData\Local\Programs\Python\Python39\lib\site-packages\pydeezer\Deezer.py", line 214, in get_track_tags
    album_data = self.get_album(track["ALB_ID"])
  File "C:\Users\nihaa\AppData\Local\Programs\Python\Python39\lib\site-packages\pydeezer\Deezer.py", line 587, in get_album
    data = self._legacy_api_call("/album/{0}".format(album_id))
  File "C:\Users\nihaa\AppData\Local\Programs\Python\Python39\lib\site-packages\pydeezer\Deezer.py", line 956, in _legacy_api_call
    raise APIRequestError(
pydeezer.exceptions.APIRequestError: type : DataException

[acgonzales]

Thanks for submitting the issue. I will look into this issue. How many tracks are you downloading using Downloader? Also, I'll be waiting for the API error.

[delrius-euphoria]

Approx. 60 songs where kept in the queue out of which 57 got downloaded. I've updated the question with the error.

[acgonzales]

Say, after downloading, does it say that all the tracks are downloaded while some of the tracks are skipped? I think I may have found the cause of the problem. I'm working on a solution.

[delrius-euphoria]

Thanks for the update, it weirdly doesn't say anything about any tracks skipped.

[delrius-euphoria]

@Chr1st-oo Hi, got any good news?

[acgonzales]

Yes, expect the commit be pushed by tomorrow or the next day. Thank you for your patience. FYI, the problem is related to the Deezer APIs not having any data about the track's album.

[delrius-euphoria]

Oh thats great, this library has been nearly perfect!! Please do let me know once its pushed.

[delrius-euphoria]

Hi, is the current fix pushed?

[acgonzales]

Sorry for the late reply, got caught up with school works. I've pushed a supposed fix e894b2a14d13bc33d83bb71d75d0f6302357b5d8 but I didn't have the time to thoroughly test it yet. Let me know if it fixes it.

[delrius-euphoria]

@Chr1st-oo Can you update pip with it too, I am having trouble pip installing the commit with pip install git+https://github.com/Chr1st-oo/pydeezer.git@e894b2a14d13bc33d83bb71d75d0f6302357b5d8

[acgonzales]

I won't be making a release to pypi unless the changes have been tested. You can easily install the new commit by cloning this repo and installing it using pip. Refer to README.md. Thanks.

[delrius-euphoria]

@Chr1st-oo Good decision, but looks like you changed the structure of data too, anyway I do get a message saying GW API error, falling back to official API.. But apart from this I am able to get the data(haven't tested downloading yet), in a different way(the indexes have changed).

[delrius-euphoria]

Some songs give out error when using get_track()['download](...), but that error is ignored with Downloader i guess, for example, try downloading https://www.deezer.com/en/track/111599572, with get_track() and then try downloading the album with Downloader, it doesn't download that song and doesn't give any error.

Traceback (most recent call last):
  File "c:\PyProjects\Music Downloader\test.py", line 11, in <module>
    deezer.get_track('111599572')['download'](download_dir='C://PyProjects//Music Downloader//dist//',quality=track_formats.MP3_128)
  File "C:\Users\nihaa\AppData\Local\Programs\Python\Python39\lib\site-packages\pydeezer\Deezer.py", line 
311, in download_track
    url, quality_key = self.get_track_download_url(
TypeError: cannot unpack non-iterable NoneType object

[acgonzales]

looks like you changed the structure of data too

Yes because the package uses two Deezer APIs (GW and official). So it tends to give different structures of data depending on which API is used. I've decided I'd map those data internally so that it gives dictionaries with the same key disregarding on which API is used.

anyway I do get a message saying GW API error, falling back to official API.

That's intended, since some data (usually the albums) are not present in GW. I implemented a fallback method so that if GW or API gives an error, it tries the other one. One caveat tho, tracks that are not in GW cannot be downloaded.

Some songs give out error when using get_track()['download](...), but that error is ignored with Downloader

I haven't implemented error handling on the most part of the package yet, specially the Downloader. Right now, it doesn't care about the errors and will just continue on the other tracks.

About the error, does this error occurs only now after the commit? Seems like the download link gives off an error 403 if accessed that's why it crashes. I'll look into it.

Thank you for the info.

[delrius-euphoria]

@Chr1st-oo This error seems new, though now, when I try pip uninstall and then pip install py-deezer the same error comes. So I guess it is old error. Anyway thanks for the update, so far so good :)

[hejops]

Any progress? I'm still getting this error without any warning message.

[delrius-euphoria]

@hejops What error

[hejops]

As you said, some tracks are not being downloaded with Downloader, and I usually don't find out about it until I start processing the downloaded files. Anyway, I wrote a crude workaround for this, which is to basically retry the download until the "correct" number of files are found in the destination directory.

def download_album(tracks: list[str], id: str, dir: str):

    if not tracks:
        print("No tracks to download")
        return

    files = [x for x in Path(dir).glob("*.mp3")]

    if Path(dir).is_dir() and len(files) >= len(tracks):
        print("Already downloaded:", dir)
        return

    else:
        Path(dir).mkdir(parents=True, exist_ok=True)

    while len(files := [x for x in Path(dir).glob("*.mp3")]) < len(
        tracks
    ):

        Downloader(
            deezer,
            tracks,
            dir,
            quality=track_formats.MP3_320,
            concurrent_downloads=4,
        ).start()

This is susceptible to an infinite loop in some edge cases, but it works for most albums in my experience.