[bounty] SSE realtime streaming of frames

[louis030195]

trafficstars

similar data structure to https://github.com/mediar-ai/screenpipe/blob/e90d426e76439120371218773588fcb32195d4d5/screenpipe-server/src/video_cache.rs#L24

https://github.com/mediar-ai/screenpipe/blob/e90d426e76439120371218773588fcb32195d4d5/screenpipe-server/src/server.rs#L1360

but real time eg as each frame come up it should stream it with OCR/UI parsing through SSE

this feature would also unlock real time API in pipes like (pseudocode):

for const frame await pipe.stream() {
  openai("what is happening in {frame}?")
}

instead of current polling

every 5 min 

data = pipe.queryScreenpipe(last5min)

openai("what is happening in {data}?")

(no need to impl this TS sdk in this issue)

this would unlock this later: #659

and next step would be to have audio + transcription too

@FractalFir do you want to work on this? I know you cannot test the UI data modality which is currently only macos but maybe i can help

let me know if need more clarification

/bounty 200

[linear[bot]]

MED-364 [bounty] SSE realtime streaming of frames

[algora-pbc[bot]]

~~## 💎 $200 bounty • Screenpi.pe~~

~~### Steps to solve:~~ ~~1. Start working: Comment /attempt #811 with your implementation plan~~ ~~2. Submit work: Create a pull request including /claim #811 in the PR body to claim the bounty~~ ~~3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts~~

~~Thank you for contributing to mediar-ai/screenpipe!~~

~~Add a bounty • Share on socials~~

Attempt	Started (GMT+0)	Solution
🟢 @EzraEllette		#1110

[algora-pbc[bot]]

💡 @EzraEllette submitted a pull request that claims the bounty. You can visit your bounty board to reward.

[algora-pbc[bot]]

🎉🎈 @EzraEllette has been awarded $200! 🎈🎊

• Share on socials
• Give feedback