Fold MPD command lists into one command to fix slow loading of large playlists

[jodal]

When I load a 2500 track playlist, no other requests get a response for a long time:

DEBUG    2011-06-14 17:06:32,565 [17829:MpdThread] mopidy.frontends.mpd.session
  Request from [::ffff:127.0.0.1]:46125: load "500C feelgood playlist"
DEBUG    2011-06-14 17:06:32,908 [17829:MpdThread] mopidy.frontends.mpd.session
  Response to [::ffff:127.0.0.1]:46125: OK
DEBUG    2011-06-14 17:06:32,929 [17829:MpdThread] mopidy.frontends.mpd.session
  Request from [::ffff:127.0.0.1]:53368: status

# Here we hang around and wait, clients timing out

DEBUG    2011-06-14 17:07:32,664 [17829:Thread-4] mopidy.backends.base
  Shuffling tracks
DEBUG    2011-06-14 17:07:34,411 [17829:MpdThread] mopidy.frontends.mpd.session
  Response to [::ffff:127.0.0.1]:53368: 
    volume: 100
    repeat: 0
    random: 1
    single: 0
    consume: 0
    playlist: 2644
    playlistlength: 2643
    xfade: 0
    state: play
    song: 55
    songid: 56
    time: 226:393
    elapsed: 226088
    bitrate: 2
    OK

[adamcik]

The direct problem here concerning tying up the old asyncore event-loop is now gone as we have glib-event-loop + threaded workers approach now.

However, there is still a noticeable delay even when loading small playlists of 250 songs. Part of the problem is once again mpd clients.

If I simply run load "playlist" via netcat everything loads nicely and only one playlist modification and change/idle event is needed. If I load the same playlist in ncmpcpp it will lookup which tracks are in the playlist and then do:

command_list_begin
addid …
…
command_list_end

This leads us to rebuild the playlist tuple once for each song - and fire the playlist event once for each addition. The same effect can also been seen when doing mpc search artist foo | mpc add.

We either need to look at how to batch up playlist changes inside command_lists or consider making the track list in the playlist mutable in order to speed this up with cheap appends.

[jodal]

mopidy/mopidy#112 is a special case of this, where in shuffle mode the playlist is reshuffled for every add in a command_list. In other words +1 for some batch processing support and -1 for making the playlist mutable, as the batch processing sounds most helpful for the mopidy/mopidy#112 special case.

[sandos]

I think this really kills my mopidy on my atom, I have a few large playlists and using them seems to sometimes "kill" mopidy.

[arybczak]

FWIW I made ncmpcpp load playlists with the load command, so at least this use case should be fine now.