gmusic-playlist.js icon indicating copy to clipboard operation
gmusic-playlist.js copied to clipboard

Imported 4965 songs, then failed

Open DavidGP opened this issue 8 years ago • 6 comments

This is amazing!

I only got the problem above when importing a 30,000 songs list. The script created 31 playlists with the naming "Thu May 26 2016 Part XY". However only parts 1-4 were fully populated. Starting from list #5, after song #4965, only like one in every 20 songs was recognized and put into the respective lists 5 to 31. I have attached the results playlist.

What could be wrong here? BTW the CPU got very hot for like 10 seconds. Maybe the script was overloaded by the number of songs?

playlists_import.zip

DavidGP avatar May 26 '16 21:05 DavidGP

that is quite a list.

searches look in two spots, in google music and in your own library. google music searches are usually quick, but don't return results for music you may have uploaded etc. library searches are not currently indexed so searches there are a bit slower.

how many songs are in your library? number found at Music Library > Songs

if you have a bunch of songs in your library (i have about 4k) you can try commenting out the line that looks in the library. processes.push(gmusic.getLibrary().then( function(slist){songs = songs.concat(slist.songs);}));

the lists are looped over in chunks to give javascript a chance to breathe and update the display. lowering the chunk size might help it process things better (although itll take longer to get through the lists). would like to have this dynamicly determined, but for now it hardcoded. look for this.chunk = 50;

I'll take a look at it this weekend

soulfx avatar May 27 '16 03:05 soulfx

Hi soulfx and thanks for your answer!

Actually I don't have any song files, only playlists. I have now started to uplad the list in chunks of like 6000. The script seems to always get until about 5000 and then the recognition fails.

What happened yesterday, it seemed that Google Music itself broke down after a couple of list uploads! Several times after the script ran, Google Music would stop working and give me a 404 server error. On another browser (without being logged into my account), the Google Play Music shop would seemingly work, but was also unable to play any song -- and gave a cryptic error message instead.

So maybe we are (the script is) somehow too fast for the Google API, with long lists...?

DavidGP avatar May 27 '16 07:05 DavidGP

Oops, I crashed Google again.

Forbidden Error 403 An unexpected error occurred. Please try again later.

Google Music, that is.

It seems that everything works when not more than 4000 songs are uploaded at a time. More than 4000, and Google Music doesn't get the additional songs, and then crashes on the next upload.

DavidGP avatar May 27 '16 18:05 DavidGP

it could be possible that google doesnt like all the simultaneous search requests from a single client

the script doesn't batch the search requests. if you load up a list of 30,000 songs without google ids it'll make 30,000 simultaneous search requests to google, and probably trigger a denial of service attack detection and mitigation routine.

soulfx avatar May 27 '16 23:05 soulfx

there was a song in the list around the 4600 mark that was causing a request loop when attempting to resolve a search suggestion that pointed back to itself. i pushed in a fix for that under eb34d41a3

with the request loop fixed and a few other improvements I was able to get in 10k songs. google does have a limit in the amount of requests it handles in a certain period of time, so I would just take the upload in small sections. if you get the 403 error, just have to wait a bit for it to reset.

soulfx avatar May 29 '16 11:05 soulfx

Thanks very much for the update!

DavidGP avatar Jun 03 '16 12:06 DavidGP