UniversalMediaServer
UniversalMediaServer copied to clipboard
UniversalMediaServer
Bookmarks to advertised folders not working in renderers which support them following UMS restart
I have used several renderers which support bookmarks (e.g. Kodi, LocalCast, VLC) in both Android and Windows implementations.
Renderer bookmarks allow the user to return to, and browse, the given folder (or sub-folder) on a given UMS server at a later time without requiring clunky navigation to get there.
I have multiple UMS servers, and it's handy to be able to quickly search the Music Videos folder (say) of each, just by clicking on a few bookmarks.
Currently (UMS 10.5), when a UMS server is restarted, its advertised "container-id" values (which are used in the bookmarks) change randomly, rendering all the bookmarks to that server invalid.
Perhaps this is a "failure to advertise correctly" issue when the server is restarted (see section 1.2 of https://openconnectivity.org/upnp-specs/UPnP-arch-DeviceArchitecture-v2.0-20200417.pdf); or, if not, maybe the container-id of advertised folders needs to be anchored somehow in one of the ini files so that they survive over restarts?
To demonstrate, I added a single shared folder "NewMusic" to UMS, and created bookmarks to it in VLC (Windows) and Kodi (Android) renderers. The bookmarks survived restarts of the renderers quite OK. I then restarted the server, and tried (unsuccessfully) to access the folder via the bookmarks.
Here are two examples extracted from the attached trace files (look for the first SOAP message containg "NewMusic"); this is from the first start after NewMusic was added to the server and browsed (container ID 53) ...
<s:Envelope` xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"> <s:Body> <u:BrowseResponse xmlns:u="urn:schemas-upnp-org:service:ContentDirectory:1"> <Result><DIDL-Lite xmlns="urn:schemas-upnp-org:metadata-1-0/DIDL-Lite/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:upnp="urn:schemas-upnp-org:metadata-1-0/upnp/" xmlns:sec="http://www.sec.co.kr/" xmlns:pv="http://www.pv.com/pvns/"><container id="53" childCount="1" parentID="0" restricted="1"><dc:title>NewMusic</dc:title><res xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/" protocolInfo="http-get:*:image/jpeg:DLNA.ORG_PN=JPEG_SM;DLNA.ORG_FLAGS=00900000000000000000000000000000">http://192.168.1.101:5001/get/53/thumbnail0000JPEG_SM_NewMusic.jpg</res><res xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/" protocolInfo="http-get:*:image/jpeg:DLNA.ORG_PN=JPEG_TN;DLNA.ORG_FLAGS=00900000000000000000000000000000">http://192.168.1.101:5001/get/53/thumbnail0000JPEG_TN_NewMusic.jpg</res><res xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/" protocolInfo="http-get:*:image/png:DLNA.ORG_PN=PNG_LRG;DLNA.ORG_FLAGS=00900000000000000000000000000000">http://192.168.1.101:5001/get/53/thumbnail0000PNG_LRG_NewMusic.png</res><res xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/" protocolInfo="http-get:*:image/png:DLNA.ORG_PN=PNG_TN;DLNA.ORG_FLAGS=00900000000000000000000000000000">http://192.168.1.101:5001/get/53/thumbnail0000PNG_TN_NewMusic.png</res><upnp:albumArtURI dlna:profileID="JPEG_SM" xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/">http://192.168.1.101:5001/get/53/thumbnail0000JPEG_SM_NewMusic.jpg</upnp:albumArtURI><upnp:albumArtURI dlna:profileID="JPEG_TN" xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/">http://192.168.1.101:5001/get/53/thumbnail0000JPEG_TN_NewMusic.jpg</upnp:albumArtURI><upnp:albumArtURI dlna:profileID="PNG_LRG" xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/">http://192.168.1.101:5001/get/53/thumbnail0000PNG_LRG_NewMusic.png</upnp:albumArtURI><upnp:albumArtURI dlna:profileID="PNG_TN" xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/">http://192.168.1.101:5001/get/53/thumbnail0000PNG_TN_NewMusic.png</upnp:albumArtURI><dc:date>2021-05-29T16:36:44</dc:date><upnp:class>object.container.storageFolder</upnp:class></container><container id="54" childCount="4" parentID="0" restricted="1"><dc:title>Web</dc:title><res xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/" protocolInfo="http-get:*:image/jpeg:DLNA.ORG_PN=JPEG_SM;DLNA.ORG_FLAGS=00900000000000000000000000000000">http://192.168.1.101:5001/get/54/thumbnail0000JPEG_SM_Web.jpg</res><res xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/" protocolInfo="http-get:*:image/jpeg:DLNA.ORG_PN=JPEG_TN;DLNA.ORG_FLAGS=00900000000000000000000000000000">http://192.168.1.101:5001/get/54/thumbnail0000JPEG_TN_Web.jpg</res><res xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/" protocolInfo="http-get:*:image/png:DLNA.ORG_PN=PNG_LRG;DLNA.ORG_FLAGS=00900000000000000000000000000000">http://192.168.1.101:5001/get/54/thumbnail0000PNG_LRG_Web.png</res><res xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/" protocolInfo="http-get:*:image/png:DLNA.ORG_PN=PNG_TN;DLNA.ORG_FLAGS=00900000000000000000000000000000">http://192.168.1.101:5001/get/54/thumbnail0000PNG_TN_Web.png</res><upnp:albumArtURI dlna:profileID="JPEG_SM" xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/">http://192.168.1.101:5001/get/54/thumbnail0000JPEG_SM_Web.jpg</upnp:albumArtURI><upnp:albumArtURI dlna:profileID="JPEG_TN" xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/">http://192.168.1.101:5001/get/54/thumbnail0000JPEG_TN_Web.jpg</upnp:albumArtURI><upnp:albumArtURI dlna:profileID="PNG_LRG" xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/">http://192.168.1.101:5001/get/54/thumbnail0000PNG_LRG_Web.png</upnp:albumArtURI><upnp:albumArtURI dlna:profileID="PNG_TN" xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/">http://192.168.1.101:5001/get/54/thumbnail0000PNG_TN_Web.png</upnp:albumArtURI><upnp:class>object.container.storageFolder</upnp:class></container></DIDL-Lite></Result> <NumberReturned>2</NumberReturned> <TotalMatches>2</TotalMatches> <UpdateID>1</UpdateID> </u:BrowseResponse> </s:Body> </s:Envelope>
... and this is the creation of a new bookmark following a Quit and restart of UMS (container ID is now 183) ...
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"> <s:Body> <u:BrowseResponse xmlns:u="urn:schemas-upnp-org:service:ContentDirectory:1"> <Result><DIDL-Lite xmlns="urn:schemas-upnp-org:metadata-1-0/DIDL-Lite/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:upnp="urn:schemas-upnp-org:metadata-1-0/upnp/" xmlns:sec="http://www.sec.co.kr/" xmlns:pv="http://www.pv.com/pvns/"><container id="183" childCount="1" parentID="0" restricted="1"><dc:title>NewMusic</dc:title><res xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/" protocolInfo="http-get:*:image/jpeg:DLNA.ORG_PN=JPEG_SM;DLNA.ORG_FLAGS=00900000000000000000000000000000">http://192.168.1.101:5001/get/183/thumbnail0000JPEG_SM_NewMusic.jpg</res><res
I've attached screenshots from the VLC and Kodi renderers giving details of their bookmarks. In the Kodi screenshot, you can see both the first (now failing) and the (new) second bookmark details.
Relevant trace files are at https://www.universalmediaserver.com/forum/viewtopic.php?p=43879#p43879
It's no "secret" that the ID's are regenerated each time, this is how the code is written. This goes back to the original PS3 Media Server.
To be able to maintain the IDs, they would have to be stored somewhere, for example in the database. UPnP AV doesn't mandate that the ID's are persistent, but strongly recommends it as far as I remember. You won't find any requirements for this in UPnP DA, and UMS (and most other things) support UPnP 1.0, not 2.0. If you really want to find it, you should look in UPnP-av-ConnectionManager-v1-Service, but it won't help anything.
It's pretty clear to me that those who wrote the standards, imagined a much more simple and rigid server than what UMS is. It also shines through that those who wrote it was interested in making renderers (the commercial companies manufacturing TVs, game consoles etc), not servers, so whenever there's a point where you have to make it easier to write either the server or the renderer, the standard makes it easy for the renderer.
UMS however generates parts of the content dynamically, depending on the content that's physically in the shared folders and the configuration (just take the entries inside the TRANSCODE folder for example). It is very difficult to understand how this could be combined with persistent IDs. UMS' code makes no such attempt, even for content that isn't dynamically generated, like files or folders. It simply assigns IDs sequentially in whatever order they are first "touched" internally in UMS' memory.
Some makers of renderers, like Kodi for example, seems to believe that the IDs can be expected to be persistent, and this causes some issues also when it comes to their caching of folder content and thumbnails.
I've been trying to figure out a way to make the IDs persistent while allowing dynamic content, it would have to be some way to make them deterministic so that the same dynamic content would always result in the same ID, but so far I haven't found a way that is "practical". I guess one could use the "path" (the combination of the element and all it's parent in the server's "tree") and then feed that into some algorithm like UUID to "normalize" this into an ID. It would result in very long IDs that would make debugging much harder, and I'm not sure what the performance impact of doing something like this would be, but I guess it's one possible way to achieve this. This isn't the way it's currently done by UMS though.
OK, thanks for the explanation. Your UUID idea sounds like a good seed idea.
Remember, to fix my problem, this only needs to be done for the advertised folders. How about a hash of the path, which is generated initially on startup ... shouldn't need to be too big to guarantee uniqueness, and it would be fast. Also, it doesn't need to be stored anywhere.
@SubJunk, I'd be happy to do that if somebody could point me in the right direction ... where are the container ID's generated?
@peterhatoz Consider yourself warned, this is a bit of a mess. I believe they are actually generated in the GlobalIdRepo
The main gotcha with the GlobalIdRepo is that the IDs get cleaned up, which is logged by the program. We made that change to use weak hashmaps because of memory use.
We certainly could use any help we can get on this project, as we don't have enough active developers to do the things we want to do, especially on fundamental things like this
Happy to contribute to such a great product. Skills are a little rusty though, so be gentle with me :-)
Also, nothing happens quickly with me any more ... retirement has ruined me :-)
Will have a look at GlobalIdRepo in the next day or so though.
No problem at all, we all just volunteer any time we can. Please let us know any questions.
It is very difficult to understand how this could be combined with persistent IDs. UMS' code makes no such attempt, even for content that isn't dynamically generated, like files or folders. It simply assigns IDs sequentially in whatever order they are first "touched" internally in UMS' memory.
Can you think of any other problems that may arise if implementing this with a database besides dynamic content? I plan to try working on a persistent storage for IDs to resolve this (I got quite annoyed by this…) and have a couple of thoughts of how to deal with dynamic content. Basically I'm planning to implement this as a persistent ID cache and not as a full storage which should give a reliable solution for dynamic content.
Can you think of any other problems that may arise if implementing this with a database besides dynamic content? I plan to try working on a persistent storage for IDs to resolve this (I got quite annoyed by this…) and have a couple of thoughts of how to deal with dynamic content. Basically I'm planning to implement this as a persistent ID cache and not as a full storage which should give a reliable solution for dynamic content.
I've been looking at this in the past myself, and there are numerous problems but I don't remember the details. I don't think there are any "theoretic" challenges except for dynamically created content, but as far as I can remember there are lots of practical problems when it comes to implementing it in the current code base. Some of the problem is that the "ID logic" has changed over time, and it used to be a string, not an integer. Back when it was a string, "sub-ID"s were build by concatenating hierarchical elements using # as far as I remember. Most of this has since been abandoned and some things are probably broken as a result, but there are bits if pieces of this code still scattered around that might still server some niche cases.
The code base is also very much based on (mostly undocumented) hacks where "special cases" are handled based on assumptions of how certain data look, like the resource ID. These things will break with any change, regardless of how "sane" the change is from a logical standpoint. So, much of the challenge is to try to track down all the "special handling", understand what it's supposed to achieve, find another way to achieve it and then implement and test it. After doing things like this long enough, I tend to lose motivation/interest, which probably is the biggest hurdle in the end.