org-noter
org-noter copied to clipboard
[Enhancement] Expand scope so that it can take notes from video and audio, both online and offline
I had this idea where you can use org-noter
to make notes from videos and audios played by a media player. It would be able to create notes and extract current timestamp of media playing in another window. I am considering making this feature, but it would take me weeks to get to speed with Emacs Lisp, and was wondering if someone else have any ideas in making this possible.
Online media content
For online media content, we could consider interacting with HTMLMediaElement, and use these particular properties/methods/events:
-
HTMLMediaElement.currentTime
property -
HTMLMediaElement.paused
property -
HTMLMediaElement.fastSeek()
method -
HTMLMediaElement.pause()
method -
HTMLMediaElement.play()
method -
timeupdate
event
A browser extension would probably be needed so that we can interact with it from outside. I can think of two approaches which I can use for passing messages b/w Emacs and browser:
- Native messaging
- WebSockets
Offline media content
For offline media content, we could consider using an existing media player such as VLC
- via
dbus
. VLC conforms to MPRIS specification. - via RC interface: https://github.com/mohkale/vlc-rc
Commands
Org Noter commands
We can utilise these following existing commands:
-
org-noter
opens media player at timestamp where user viewed video/audio last -
org-noter-insert-note
inserts heading including timestamp in:PROPERTIES:
drawer, down to seconds resolution -
org-noter-insert-precise-note
inserts heading including timestamp but in milliseconds resolution -
org-noter-sync-current-note
jumps to timestamp at current heading -
org-noter-sync-next-note
jumps to timestamp at next heading -
org-noter-sync-previous-note
jumps to timestamp at previous heading -
org-noter-kill-session
kills session, which also closes media player
All other commands are unsupported for this MVP.
Media player commands
Additional commands are needed for interacting with media player within Emacs. I can think of a few so far:
-
org-noter-media-jump-to-timestamp
Prompts user to type in timestamp to jump to -
org-noter-media-toggle-play
toggles play/pause media - seek back or forward slightly
Does this software already exist?
To my knowledge, there is no org note taker for videos/audios. This section from "How to annotate literally everything" article says that no such note taking software exists.
These links are relevant and may be able to get this feature working:
- Bi-directional communication using WebSockets: https://github.com/alpha22jp/atomic-chrome
- Edit subtitles for video: https://github.com/rndusr/subed
- Video command primitives: https://github.com/skeeto/transcription-mode
- Video editor for Emacs: https://www.emacswiki.org/emacs/GNEVE
- org-protocol: https://orgmode.org/worg/org-contrib/org-protocol.html
Example
* Big Buck Bunny - Blender Short Film - YouTube video
:PROPERTIES:
:NOTER_LINK: https://www.youtube.com/watch?v=aqz-KE-bpKQ
:NOTER_CSS_SELECTOR: .video-stream
:NOTER_TIMESTAMP: 2:40
:END:
** Bunny picks up fallen apple
:PROPERTIES:
:NOTER_TIMESTAMP: 1:28
:END:
** Credits
:PROPERTIES:
:NOTER_TIMESTAMP: 8:15
:END:
I had almost the same idea last week, though I'd focus on offline video
I'd use the mpv player with some modification, so you can press a keybind to cature the position/area of a video and annotate it in emacs
Since mpv is easily scriptable, this should be the easiest way.
I guess as you can open web videos in mpv as well, online functionality comes for free as well :)
https://github.com/kljohann/mpv.el/wiki
see also https://github.com/yuchen-lea/org-media-note
MPV sounds like the best media player to use for this use case.
From https://github.com/weirdNox/org-noter/issues/82#issuecomment-542424247, I don't know about hooks, but it seems like if I can use these hooks to add support for video and audio files, then I can build on top of it and create a new repo called org-noter-media
which will incorporate ideas I have above as well as a synthesis of ideas coming from alternative packages. If this is not a good approach (like what the author said regarding content that does not have pages), then a completely separate package will be better, and this issue will be closed.
Hooks can be found here: https://github.com/weirdNox/org-noter/blob/9ead81d42dd4dd5074782d239b2efddf9b8b7b3d/org-noter.el#L207-L210 https://github.com/weirdNox/org-noter/blob/9ead81d42dd4dd5074782d239b2efddf9b8b7b3d/org-noter.el#L226-L271
Hooks are being used from org-pdftools
package: https://github.com/fuxialexander/org-pdftools
https://github.com/fuxialexander/org-pdftools/blob/a5b61bca3f8c91b0859bb0df1a929f9a31a57b99/org-noter-pdftools.el#L264-L274
I'd like to inform everyone that I implemented a working prototype which depends on org-media-note and mpv.el borrowing from @auroranil's idea here. For it to work, I have to modify org-noter to accept a link as a document source so it only works with my fork of org-noter, org-noter-plus-djvu.
For how to install read its README here.
May be a good idea to migrate this to https://github.com/org-noter/org-noter
- https://github.com/weirdNox/org-noter/issues/174
I have no plans for this project. Feel free to fork this project and make changes to it.