mediar-ai
timeline ui
i have been reticent for a while on this feature because screenpipe is not intented to be a copy of rewind or things like this and more focused on infrastructure / devtools
but this is more intuitive
https://screenmemory.app/
https://www.rewind.ai/
i guess all that is required is:
- [x] merge mp4 endpoint
- [ ] plumbing with audio
- [ ] global shortcut
- [ ] (later) select part of your day to prompt AI
- [ ] maybe: annotate times with LLM/pipes
- [ ] eventually all pipes could be somehow integrated with this e.g. "enrich my CRM pipe" would show somewhere i had conversation with customer john and click bring to salesforce or idk
- [ ] ?
![linear[bot] avatar](https://avatars.githubusercontent.com/in/20150?v=4 "Published by a pub.dev verified publisher"){kind=small}
but this is more intuitive
My 2c here is that this adds the ability to 'sense check' the LLM response, which is excellent. I'm using Reclaim and saw this as a near drop-in. I was just about to purchase it, but then I noticed that the timeline was missing.
Understanding the reclaim.ai has also pivoted to the pendant may create a market gap as well.
bounty for a small part of this, e.g. just showing a scrollable timeline above current app i'm using even if it's full screen
definition of done:
- suggest some designs (excalidraw, figma, whatever) and Ill validate
- when i do specific shortcut from any app, it shows a scrollable timeline (can be fake data initially, i can do the plumbing with .mp4)
- works even if i'm in a full screen app
- works on macos and windows at least
Timeline view with Rewind was one of my favourite and most used features. Would be a must have for me to switch from Rewind to screenpipe completely. Made it super quick to look back in time to see my screen, copy text (you already have OCR ✅ ), etc.
biggest blocker for this is how to show window on top of full screen app in tauri?
how to show window on top of full screen app in tauri?
If you're looking to emulate the Rewind.ai app then this Is understandable.
That said if I was to make a user story, I'd say:
As a user, I want to view the mp4 history with the timestamp to align it with the AI responses. This will enable me to check the response and eyeball what's on the screen if the LLM provides a subpar result.
As for the UX I would not mind if it were another window with the same ratio as my screen yet smaller (or even resizable), and I could zoom in if need be. As long as I can 'change to that window' and focus on it while viewing, I would not mind.
Tauri, as your platform, will have pros and cons. This may be a time when speed of development is prioritised over 'the perfect UI'.
@Glavin001, does the above resinate with you?
an idea i had also but kinda different is to allow user to scroll horizontally the video in results here
but i think the main request for this timeline ui is to be able to search, fast, anywhere, while now it's more friction to get more precise results
so timeline to my understanding is more a low friction, quick result and the scrollable timeline feels quite natural compared to a complex search interface
also if we ship multimodal embedding we could do multimodal semantic search which could be different / interesting UX