MPD
MPD copied to clipboard
MusicPlayerDaemon
support multiple music_directory paths
Feature request
MPD appears to support a single music_directory path. Enabling additional directories via multiple music_directory calls would be a convenient feature.
A single music_directory works only on the assumption the user keeps their music in a single location. Some users may have their music on multiple drives, including removable ones, which they may be connecting at any time when MPD is already running.
Failure to find the directory at start time should not be taken as assumption the path won't ever become available during the lifetime of the MPD process.
The simplest use case for missing paths at start time is for use with removable media, which sit in mountpoints under /media/<username> (controlled by udisks2), which may not be present if no removable media is inserted. It would be up to the user to scan the media, so polling for changes (with auto_update) need not apply to the directory.
To clarify how it would work: a user could set their local music_directory and add a second music_directory clause pointing to /media/<username>. The former would work with auto_update, whilst the latter path would be scanned for files whenever the user specifically requests MPD to do so.
How would this work, at the protocol level? Do you know that MPD can integrate with udisks2 to integrate removable media into its database?
Reading through the code, I've just stumbled upon the udisks2 plugin. Does it support any media inserted, by specifying udisks://, without the label or name? If the label of the media has to be known upfront, it may be difficult to use without access to the MPD configuration (other than a rescan option, which again may be automated by including udev rule whenever a user inserts or removes removable media). Users expect to insert and remove media without making configuration changes every time a new one is added.
At protocol level, supporting multiple paths, with non-existent ones at mpd start time, may require three changes, if there isn't already code present to enable those:
- MPD needs to virtualise all music_directory paths provided in a common structure. Be aware some music may be available locally, and some sit on removaable media. A user will want to access both. Failure to find a path should prompt for MPD to record it for a subsequent rescan.
- Auto_update needs to be aware of path type at any given time. Any non-existent paths are assumed to be unpollable and excluded from auto_update. If a path becomes missing and unpollable, such as caused by removing a usb stick, auto_update ignores it until a rescan.
- At user requested rescan time, MPD attempts to walk all the paths including non-existing ones. Existing paths that become non-existent are flagged as non-existent and their music directory contents removed until a future rescan. Failure to access a path prompts MPD to add it to the non-existing path list, pending a future scan.
The reason why considering non-existing path is needed is simple: without any media inserted, /media/<username> is missing, so it is not possible to use /media/<username> as a generic placeholder for all removable media, without seeing the message "exception: Failed to access /media/<username>: No such file or directory".
Reading through the code, I've just stumbled upon the udisks2 plugin. Does it support any media inserted, by specifying
udisks://, without the label or name?
No. You can obtain a list of available media using the listneighbors command, see https://mpd.readthedocs.io/en/latest/protocol.html#mounts-and-neighbors because udisks is both a storage and a neighbor plugin, see https://mpd.readthedocs.io/en/latest/plugins.html#neighbor-plugins
At protocol level, supporting multiple paths, with non-existent ones at mpd start time, may require three changes, if there isn't already code present to enable those:
None of those 3 changes describe how this works at the protocol level, because none of this is about the protocol.
When you do "lsinfo", which music_directory will be displayed?
It wasn't clear it was MPD internal protocol that you meant.
music_directory "/media/<username>"
music_directory "/misc/local"
lsinfo displays whichever directory is configured last as music_directory. Any earlier directory does not appear to be looked at. From the example above, /misc/local comes last so that's what is examined for music. /media/<username>, whether present or not, won't be examined. Granted, if the order were reversed, i.e.
music_directory "/misc/local"
music_directory "/media/<username>"
the user's removable media directory coming last, it would be examined at a user initiated rescan, even if originally missing at MPD start time.
Still, only a single path is supported. True, protocol may require some constructs for adding/removing paths. To extend the present syntax, for adding & removing paths, the following config file parameters could work:
music_directory+ <path>
music_directory- <path>
That way, the compatibility with the existing behaviour of music_directory, where the last parameter provided overrides earlier ones, would be maintained.
I just tested the udisks neighbor plugin and it seems to work fine. I guess the only issue related to this is the static nature of the current approach. This could be solved if the clients themselves implement the neighbor and mounting functionality. But this still puts a burden on the user to manually add it every time and the client.
One could write a separate MPD client to deal with neighbor events and then just mount them and do some action (see blocker here https://github.com/MusicPlayerDaemon/MPD/issues/1270). I guess this is the least invasive solution. Otherwise, MPD itself could provide this auto-mount functionality. But I do not know how it works with the other neighbor plugins.
And then there is the issue about what should happen if you have songs in the queue or some playlist and unplug it (see https://github.com/MusicPlayerDaemon/MPD/issues/1142).
All of tgis
I just tested the udisks neighbor plugin and it seems to work fine. I guess the only issue related to this is the static nature of the current approach. This could be solved if the clients themselves implement the neighbor and mounting functionality. But this still puts a burden on the user to manually add it every time and the client.
One could write a separate MPD client to deal with neighbor events and then just mount them and do some action (see blocker here #1270). I guess this is the least invasive solution. Otherwise, MPD itself could provide this auto-mount functionality. But I do not know how it works with the other neighbor plugins.
And then there is the issue about what should happen if you have songs in the queue or some playlist and unplug it (see #1142).
Clients typically lag behind in functionality and tend to overlook features overly complex to implement or such that users won't ever look at.
While a udisks plugin certainly showcases functionality of MPD, it won't be used much unless you are in the position to perform media discovery, uuid look ups only to mount a usb drive. Not many users will. Many (like myself) will depend on a set and forget configuration.
This request isn't about removable media anymore. It's up to the user what directories to set up for MPD. The requested feature would benefit any use case with music in multiple directories.
The existing removable media mounting mechanism in most OS distributions is perfectly fine for the job of mounting media. No plugins are strictly necessary to support adding/removing multiple media. Let KISS prevail here. Many unnecessary problems mentioned in your post could be avoided, had there been support for multiple directories in MPD.
Clients typically lag behind in functionality and tend to overlook features overly complex to implement or such that users won't ever look at. [..] This request isn't about removable media anymore. It's up to the user what directories to set up for MPD. The requested feature would benefit any use case with music in multiple directories.
I would assume since udisks2 is also available as storage plugin you should be able to set it up in the configuration exactly as you intend to. There is just a little bit of documentation lacking there.
I would assume since udisks2 is also available as storage plugin you should be able to set it up in the configuration exactly as you intend to.
No, I'm not aware the udisks plugin works that way. You need to list the media manually with one plugin then mount them or input the device UUID upfront. Too much hassle if the goal is to access the media already automounted under /media by the OS.
There is just a little bit of documentation lacking there.
Indeed, the non-existent documentation is poor encouragement for the layman to try them out. I did and these plugins are impractical in an static configuration environment, with limited access by users.
Most clients support rescanning so users can perform that step easily themselves without raw access to the obscure interface that plugins currently use. The OS is doing the automounting just fine. All that is missing is support for multiple music_directory locations (with one of them being /media in my use case).
Given the multitide of options in MPD, partitions, multiple music outputs, etc., I was surprised NOT to find support for multiple music directories.
Correct me if I’m wrong, but can’t you just mount all dynamic mountpoints inside the configured music_directory, or alternatively somewhere else and add symlinks inside the music_directory pointing to the mountpoints?
The same works for multiple directories in general.
On changes you just trigger a rescan as you already suggested.
Correct me if I’m wrong, but can’t you just mount all dynamic mountpoints inside the configured music_directory, or alternatively somewhere else and add symlinks inside the music_directory pointing to the mountpoints?
It's not ideal, it won't work with nfs specific music_directory. Nfs access actually works better in mpd than over a static mount or an autofs, which still suffers from issues, like need to restart the service after a failed attempt to resolve a server name initially or recovery problems. Multiple music_directories give the flexibility to pick any type of sources, be it a path or a plugin, mixing them as needed.
Also, if a linked to directory is ever moved you need to remember to update the links instead of mpd configuration.
Is it not possible to create a symbolic link to achieve the same effect? Say create one directory that is called /Music/, which contains symbolic links to all other multiple directories that have music.
Music/
symlink1_pointing_to_music_dir_1
symlink1_pointing_to_music_dir_2
symlink1_pointing_to_music_dir_3
and then inside of mpd.conf have music_directory "/Music/" follow_outside_symlinks "yes", and follow_inside_symlinks "yes"
No, it's not the same effect. First of all, you need file system level access in order to be able to do that. Consider working in a limited shell scenario. On the other hand, configuration access may be granted to a single read-write mpd configuration file, but nothing else. Second issue and I think that may be the crux that it does not support mixing plugin music_directory sources (such as nfs://) with a file system-based one. Third, it's a poor man's hack, concealing the fact that mpd does not support multiple music sources at the same time...
I understand. Maybe someone who is using mpd on a personal computer with music stored in multiple directories (including external harddrives) might find this hack useful in the meantime.
Yes, that is what I meant, too. It works well for physically attached storage. It also works with NFS and other stuff that can be mounted on the system level, but as @twojstaryzdomu pointed out it is rather suboptimal.
At least for NFS and other protocols MPD supports the client protocol mount command seems like a good solution to me. Afaik, the mounts are not persisted during MPD restarts, or am I wrong? That would be an important addition I guess.
As for the udisks2 support it might be an idea to make the drive identifiers more useful. Iirc they are not that human-friendly.
Edit: Sorry if some things were answered already. I didn’t see the new responses.
Indeed, @zoj603, you're right as rain and I'd thought of your suggestion as a possible solution before raising the issue.
MPD provides an impressive set of options that extend quite a bit beyond the basic player and yet it falls short in this instance to provide a simple parameter to pick more than one music source.
@twojstaryzdomu have you tried the mount command so far?
I suppose you mean the internal mount command that for instance is mentioned in #1270?
I'm reluctant to use that since the mount command is internal to MPD but it isn't part of the configuration. Given the listing capabilities, it's a great facility for a future client to interact with MPD, but I don't see that having an equivalent in a config file. At least one that is mentioned in the mpd.conf manpage.
mount, list are great for interactive or programatic use, like in the client that you are developing. These commands address a different set of challenges to a suggested static mpd.conf option, defining multiple music_directory sources.
Yes, i mean that. However it is not internal, otherwise it could not be used via the protocol. mpc supports it.
I would suggest to try it out even if it has no place in the config yet. Because if it works well for your use case you can advocate for adding it to the config. And adding mount to the config seems much more likely than adding something completely new, imo.
Just be sure to configure the cache_directory, because afaik that is necessary for mount to work.
IIRC internal commands are no good match. mpd.conf is parsed only when mpd starts. There isn't really any need for mount, only the ability to use multiple paths.
I think the complexities of the listing & mounting of drives outweigh the simple scenario merely to be able to specify multiple paths in the mpd.conf file. There isn't any need to extend the internal commands.
One way to handle multiple paths would be to extend the syntax for the parameter music_directory in mpd.conf to include multiple paths, separated by a semi-colon. That way, it would remain backwards compatible with all earlier setup, all the while allowing newer clients to take advantage of the extension.
@twojstaryzdomu why do you keep calling it internal? What makes it internal?
And like I said above, I would be fine already if mounts were persisted in some way, if they are not already. I just said that adding mounts to the config would be an idea since you said that you want some config way.
@maxmitti, I take back that proposal.
extending client-level commands merely to be able to support multiple paths that you specify in a static mpd.conf file sounds overkill.
mount doesn't sound like it has anything to with multiple paths anyway. It's a client-level command. The fact you need to add some cache_directory earlier defeats the simplicity of the exercise.
Config file parameters need to be self-sufficient, not depending on one another. That's one reason why config file parameters can't be a good match for client-level commands.
Over-engineered solutions tend to be ignored in favour of simpler ones. Invoking an obscure set commands is perfectly fine for client protocol but not a good idea when it comes to config files that need to be simple for users' sakes.
If all the users need is to be able to specify multiple known paths in mpd.conf, I'd suggest extending music_directory to include paths separated by a semi-colon. That solution follows KISS.
Besides, mpd clients tend to lag behind in protocol changes that aren't core functionality. If the set of paths ever changes, it's by far easiest to update mpd.conf and restart mpd. Doubtless, that will work with all the clients.
I don’t get what your problem is. Configuring the cache_directory is a one time thing. Especially if specifying mount points would be done via the config that would be done in one go anyway.
@maxmitti, I take back that proposal. extending client-level commands merely to be able to support multiple paths that you specify in a static mpd.conf file sounds overkill. […] It's a client-level command.
While I was also talking about adding config options to specify mount points in the config, I would not necessarily say that it is an extension of a client command. It would be merely a second way to use the mounting mechanism, which just happens to be only exposed as a client command currently. I am just calling it mount command now, since that is how it is currently known.
mount doesn't sound like it has anything to with multiple paths anyway.
If you are concerned about the naming of the config value, I would guess that there can be found a name that is more intuitive. At least I didn’t suggest any specific name yet.
Over-engineered solutions tend to be ignored in favour of simpler ones. Invoking an obscure set commands is perfectly fine for client protocol but not a good idea when it comes to config files that need to be simple for users' sakes.
I don’t think my proposal would be considered over-engineering, since it is only adding another way to use existing behavior. (As I have said above, it is not about extending a command. It is about using an existing mechanism.)
One of the main advantages of reusing the mount mechanism would be that it is clear how the different directories integrate with each other, because mount points are specified as a directory in the main music_directory, i.e. there is no question which directory to use for example with lsinfo, as @MaxKellermann asked.
Another main advantage would be that it is probably very little work to implement it, because the mounting mechanism is already fully implemented.
Besides, mpd clients tend to lag behind in protocol changes that aren't core functionality. If the set of paths ever changes, it's by far easiest to update mpd.conf and restart mpd. Doubtless, that will work with all the clients.
This argument is completely irrelevant. Mount points work completely fine without specific client support already (which you would have seen if you at least tried it once). Client support is only needed currently to specify mount points, but would be completely unnecessary if they can be specified via config instead. And just as you say it would be a config change and restart as well when changing paths. However, additionally it could also be done live via the client protocol without any need for restarting.
Fine, how it's implemented under the hood is irrelevant to a user as long as it is simple and self-contained. If it's implemented in the wrong way, you clutter up the software with bad design and the feature won't see much use due to complexity.
If support for multiple directories is still going to take the literal mount with some cache_directory prereqs then it does not meet the principle of KISS.
If the simple ability is going to be exposed to the user in a user-friendly way, by way of a single mpd.conf parameter, then perfect.
By analogy, think of how cumbersome would be to have say the ls command with special prereqs, whenever you want to list more than one directory. Luckily, ls designers understood what is essential to good design. Secondary group records in /etc/group are an equivalent example to my proposal how to expose multiple paths support to users.
Would the solution you have in mind be capable of extending music_directory's syntax with multiple semi-colon separated paths? Again, that's only one suggested way of going about the task in a user-friendly way.
Wrapping around plugin code is only fair to accommodate users who just wish to specify an additoinal path (or any number) without going through the nitty-gritty of mounting, listing etc.
I would let someone who is well aware of MPD’s code decide whether it is KISS or not.
Currently the cache_directory is needed (see https://mpd.readthedocs.io/en/latest/user.html#mounting-other-storages-into-the-music-directory). Maybe one could find a reasonable default value (implicitly or in the default config). However it’s not a big issue in my opinion, because the necessity would be mentioned in the doc anyway.
In theory it would be possible as an extension of music_directory, however I it seems sub-optimal to me, because you would still need to specify in which directory the mount point should be. I think a different syntax would be more appropriate, however it is called. I could imagine specifying each directory as a group item with target and url settings, similar to how audio_outputs are configured. (As usual, the naming should probably be different, I am just making something up for examples.)
external_directory {
target "foo";
url "nfs://bar/foo";
}
Or if a completely new config syntax is accepted, it could be specified as some kind of map of target directory => url mappings.
external_directories {
"foo" => "nfs://bar/foo"
"baz" => "http://foobar/baz/"
}
Edit: While the map syntax looks much neater, I would actually go for the group syntax, because it would make it possible to add more settings in the future. For example, for specifying to use a proxy database for that directory. (I am still hoping for the addition of that feature.)
In theory it would be possible as an extension of music_directory, however I it seems sub-optimal to me, because you would still need to specify in which directory the mount point should be.
And why exactly do you need to specify the mountpoint? You certainly don't need to for music_directory! Isn't this an example of over-engineering or feature creep that isn't requested at all here?
Think about what users need this for. They want all of their music, from multiple sources, made available within a single hierarchy exposed by mpd. They'll likely won't care where that sits so long as they can readily pick it from an mpd client. A client need not even know under which mount a song is found, provided that mpd does the fair job of providing a handler for that.
I somehow don't think there is any need for mpd to be a file system manager for music, exposing the exact hierarchy based on mount sources. The existing hierarchy exposed in mpd as artists/albums/song names/other tag properties is quite enough.
For the ability to specify paths, whose mountpoint in mpd name is largely irrelevant, I'd just go with the simple semi-colon separated or rely on music_directory1, music_directory2, etc.
The need to specify the mount point is simply to keep everything else working as is. Specifically it answers the question which directory to pick as the “root” (see https://github.com/MusicPlayerDaemon/MPD/issues/1252#issuecomment-908452071).
I have seen your answer, proposing to just use the first or last in the config file, however that is not a suitable solution in my opinion.
You certainly don't need to for music_directory!
Yes, because it is the only music directory which should obviously also be the root.
You might be happy with the tag based database view, but there are many users (including me) who use the actual folder structure view.
I get that it is an extra step to do for the user, but you can consider it as a label for that folder, instead of some actual folder hierarchy. Furthermore, we are talking about people who specifically setup a client-server-model music player, it is not a choice that people who want the (setup-wise) easiest possible music player make. And it only affects those who actually need that feature, the basic single music_directory setup would stay the same anyway.
And I still don’t think that it would be over-engineering, because it actually makes the implementation simpler and already works like that.
It's
I get that it is an extra step to do for the user, but you can consider it as a label for that folder, instead of some actual folder hierarchy.
Mountpoint name is clutter, does not serve any purpose in the context of mpd.
Simpler implementation does not rule out over-engineering. The mount plugin itself a is an example of cluttering up mpd with a feature set not traditionally associated with a music player. OS based automounters have been around for much longer than mpd and are more fitted for the job. I understand your wanting to piggy-back on that functionality but while it makes sense for an informed user such as yourself, most won't even bother about mountpoint. Time is wasted explaining why you'd want to specify one in the first place, merely to be able to add another music source.
Users likely won't care there is a plugin underpinning support for multiple music sources.
So let's do that stuff under the hood and not encumber the user with specifying items that aren't strictly necessary (like mountpoint name) when specifying more than 1 music source. Not doing so runs the risk of feature creep and making the simple configuration files unnecessarily complex.
Besides no other playback software that I know of requires you to pick a mountpoint for a music source directory.
How it is implemented internally is irrelevant, what matters is how it's exposed to a user.
I get that it is an extra step to do for the user, but you can consider it as a label for that folder
Don't force users to select labels that aren't strictly necessary for the software to work. If they choose to do so voluntarily, fine, but no value here for most users wanting a shared hierarchy of all of their music on different media.
Yes, because it is the only music directory which should obviously also be the root.
Would wrapping around all the music found under all the sources to make it appear as if all were in root be a solution? No mountpoints necessary, all in line with KISS. That way forward would favour the simplest 2ndary group-style record for music_directory.
However, if a user should choose to maintain label/mountpoint per music source, fine, use the map syntax.
The reason being the disconnect to the syntax for a single path (music_directory) and multiple paths (maps, mountpoints) is uncanny. It'll no doubt raise some eyebrows if implemented the wrong way.
Pulseaudio is a good example of bad decisions and missing features leading to users leaving in droves for simpler audio engines (hopefully...). Only recently did PA expose a view of the default sink, volume level etc. with get- query commands. At a time when PA is slowly becoming irrelevant, for various reasons. While I don't claim mpd is anything close to the state pulseaudio is these days, I'm an advocate for simple solutions, wrapped around internally to accommodate users without a great need to read through plugin manuals to use a simple feature.
The mount plugin itself a is an example of cluttering up mpd with a feature set not traditionally associated with a music player.
Similarly, supporting file access via NFS (and other protocols) is also not traditionally associated with music players. Anyway, I would say that MPD is not a traditional music player.
OS based automounters have been around for much longer than mpd and are more fitted for the job.
Yet you were not satisfied when I suggested OS level mounts. However, I agree that it is not optimal for non-local storage.
How it is implemented internally is irrelevant, what matters is how it's exposed to a user.
Of course the user doesn’t care about implementation details, but the maintainer probably does.
That way forward would favour the simplest 2ndary group-style record for music_directory.
That might be an idea, but the question arises what happens to conflicting (matching name / relative path) folders and files.
Similarly, supporting file access via NFS (and other protocols) is also not traditionally associated with music players.
NFS, Samba make sense since they act as a music source similar to cloud based source. And players do have support for those.
Anyway, I would say that MPD is not a traditional music player.
The problem starts when run-of-the-mill features are missing and priority is given to clutter. Handling multiple paths is nothing special for audio players, whilst mounting internally via a label is.
That might be an idea, but the question arises what happens to conflicting (matching name / relative path) folders and files.
Does the user care what the label for their source is when all they want a single hierarchy view? Likely not. Just handle that internally somehow, use the full path for it and the problem is solved. mpd is a music player, some choices fitting in a file manager won't be relevant here. What matters first and foremost is that music can be played and easily viewed from a single hierarchy. If you argue some users will need separate hierarchies, then it follows the example of partitions, nice in theory but a minority feature. I don't think it would be used much.
Multiple media with music are in contrast a common occurrence, hence the need to add the feature. A single music_directory source only follows an idealistic assumption of an organised developer. Single use case. Currently, the usage of an nfs:// source with music_directory rules out a local one and vice versa. Users aren't like that, for many reasons. Limited storage is a good argument why.
OS based automounters have been around for much longer than mpd and are more fitted for the job.
Yet you were not satisfied when I suggested OS level mounts. However, I agree that it is not optimal for non-local storage.
The point was not whether mount is unnecessary in mpd. It may be fine internally for some things. It seems like a bad design idea to copy its interfaces as is (i.e. requiring a mandatory label) to implement multiple paths. Hence the idea to wrap around it in a smart way.
In a music player, you don't need a mount label. Think what you see in the client. You see artist/album/song/other tag. If you're truly intent on knowing the source a song comes from, add a tag that represents the source. Just don't require additional variables (mountpoints/mount labels) for a user to maintain in the configuration.