uosc
uosc copied to clipboard
tomasklaen
Thumbnails in timeline
Display thumbnails when hovering the timeline, similar to https://github.com/TheAMM/mpv_thumbnail_script
Yes I do wish adding this at some point, but I expect that to have a long ETA.
I did a small writeup on the state of this request in the other issue, but for completeness and everyone that is subscribed to this one, I'll copy it here as well:
no longer relevant wall of text used to be here
So here's the current stance:
I've researched existing solutions, and tried to think of some myself, and they all suck. They are either super slow to query the thumbnail, or alternatively they encode the whole file into hundreds of thumbnails to a drive on opening a video, so that querying is faster later (when someone has already decided what part to watch and doesn't need fast thumbnails anymore). This causes high CPU usage and laptops flying away every time someone opens a video.
So unless mpv adds an API that will allow us to query and display a thumbnail without the need to spawn child processes and re-encode the video for a minute, I won't be adding anything like that into uosc.
However, if someone makes a thumbnailing script with API where I can send it a message to display a thumbnail of a queried time at configured position, and another one to hide it when cursor leaves timeline, I won't mind doing that so that masochists who don't mind these thumbnailing solutions can use uosc too.
The only alternative solution I kind of liked would be to create and distribute a binary with uosc that would spawn an efficient thumbnailing server and than tell it to generate thumbnails on demand. But even if I'd found a time to make it, I'm not sure if we can spawn and communicate back and forth with a live child process from mpv's lua environment.
I have been looking into this and have a working proof of concept for thumbnails basically as fast as keyframe seeking, on the fly with low latency and decent memory usage (relative to existing implementations that spawn many mpv processes), for Linux and Windows.
(ugly display for testing)
https://user-images.githubusercontent.com/42466980/190234102-8b300b56-49e2-4537-9bcb-038e1ab4c52f.mp4
I agree that it should be its own script, with a way for others to order it around. I hope to have something worth publishing in the not too distant future.
I have been looking into this and have a working proof of concept for thumbnails basically as fast as keyframe seeking, on the fly with low latency and decent memory usage (relative to existing implementations that spawn many mpv processes), for Linux and Windows.
Sounds great, how does it work? Will it also work with network streams? I doubt it's possible without any extra network usage.
@po5 I'd also love to know how it works. If it's efficient and doesn't consume any resources when user is not actively seeking, and doesn't flood the drive with hundreds of thumbnails, I'd use this myself :)
It can (read: does) work with network streams, but does require its own network cache, I don't think it's worth it considering the memory, latency and bandwidth implications.
As far as I understand it, yt-dlp can expose video thumbnails (even without an extra call from a script, with the right mpv options). This should be used for YouTube and other services that support thumbnails.
It spawns a hidden paused mpv in encoding mode with the same file, that gets keyframe seek commands from the ipc. The background instance takes up much less memory than the original, and can be killed after some inactivity period.
Two files are created: one for the ipc socket (virtual on windows), one for transferring requested thumbnail across processes, it gets overwritten whenever a seek happens (apparently two-way mpv pipes do not work on windows). No extra temp cache.
If you want to discuss development and not flood this issue, you can reach me on Discord at https://gist.github.com/po5/ba7f443ea90080b2b5ccee7ff7b79023/raw/bb424e0323090942cd27133d69e75f09de001810/discord.txt
Would be glad to make a group chat to throw ideas around.
It can (read: does) work with network streams, but does require its own network cache, I don't think it's worth it considering the memory, latency and bandwidth implications. As far as I understand it, yt-dlp can expose video thumbnails (even without an extra call from a script, with the right mpv options). This should be used for YouTube and other services that support thumbnails.
One last idea (maybe you're already doing that).
When there are no thumbnails available from ytdl, you could use --format=worst to save bandwidth or maybe some combination of format and format-sort to get the video resolution that is closest to the displayed thumbnail.
Finally figured out the race condition that made the player crash sometimes when rendering the overlay.
I'll be able to work on making it look nice and add QoL stuff now. Here's hoping for this weekend.
https://user-images.githubusercontent.com/42466980/190829265-82dea8ee-414e-475a-8ae0-ac94304522fe.mp4
