[bounty] Windows UI Automation

[louis030195]

definition of done

• use windows rs lib to implement ui monitoring just as it's done on macos using accessibility API
• can query etc through app

/bounty 200

https://github.com/microsoft/windows-rs/blob/7ae4d4483e5fd1287cd43f2f352ba72753590d8f/crates/samples/windows/uiautomation/src/main.rs

[linear[bot]]

MED-295 [bounty] Windows UI Automation

[algora-pbc[bot]]

💎 $400 bounty • Screenpi.pe

Steps to solve:

1. Start working: Comment /attempt #703 with your implementation plan
2. Submit work: Create a pull request including /claim #703 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to mediar-ai/screenpipe!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @varshith257	Nov 20, 2024, 3:12:25 AM	WIP
🟢 @Raxephion	Dec 6, 2024, 9:37:43 AM	WIP
🔴 @Rutik7066	Feb 28, 2025, 7:24:48 AM	#1696
🟢 @tribhuwan-kumar		#1694

[varshith257]

/attempt #703

Algora profile	Completed bounties	Tech	Active attempts	Options
@varshith257	1 mediar-ai bounty + 19 bounties from 9 projects	Rust, Scala, Go & more		Cancel attempt

[varshith257]

Done cool progress on it! Just need to re-evaluate and push PR

[Raxephion]

/attempt #703

Options

• Cancel my attempt

[Gyan-max]

hii @Raxephion @louis030195 @varshith257 any update on this issue?

[rutikthakre]

/attempt #703 We can use https://github.com/AccessKit/accesskit AccessKit makes it easier to implement accessibility, for screen readers and other assistive technologies, in toolkits that render their own user interface elements. It provides a cross-platform, cross-language abstraction over accessibility APIs, so toolkit developers only have to implement accessibility once.

It Provides the adaptor for macos ,windows and unix platforms https://accesskit.dev/

Algora profile	Completed bounties	Tech	Active attempts	Options
@Rutik7066	1 mediar-ai bounty + 11 bounties from 8 projects	Go, TypeScript, Rust & more		Cancel attempt

[louis030195]

i'm going to share how we want this designed next week, no point in working on it before

[rutikthakre]

@louis030195 That's great! I am looking forward to implementing the UI automation. Till then, could you please assign the issue to me. So it will be easier for other folks.

[tribhuwan-kumar]

i'm going to share how we want this designed next week, no point in working on it before

sure!

[rutikthakre]

@louis030195 is their any updates on the design?

[Raxephion]

Goodmorning,

Just saw these mails now - my apologies as I was booked off sick for the last couple days and wasn't applying myself. I am, however, back in the office today. Will pick up loose ends and see where it goes and status quo

On Sat, 21 Dec 2024 at 18:52, Gyan Ranjan @.***> wrote:

hii @Raxephion https://github.com/Raxephion @louis030195 https://github.com/louis030195 @varshith257 https://github.com/varshith257 any update on this issue?

— Reply to this email directly, view it on GitHub https://github.com/mediar-ai/screenpipe/issues/703#issuecomment-2558172424, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGFTJFUPGTMPCZKLSYASEQD2GWMEBAVCNFSM6AAAAABSDNEWQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJYGE3TENBSGQ . You are receiving this because you were mentioned.Message ID: @.***>

[rutikthakre]

Goodmorning,

Just saw these mails now - my apologies as I was booked off sick for the last couple days and wasn't applying myself. I am, however, back in the office today. Will pick up loose ends and see where it goes and status quo

On Sat, 21 Dec 2024 at 18:52, Gyan Ranjan @.***> wrote:

hii @Raxephion https://github.com/Raxephion @louis030195 https://github.com/louis030195 @varshith257 https://github.com/varshith257 any update on this issue?

— Reply to this email directly, view it on GitHub https://github.com/mediar-ai/screenpipe/issues/703#issuecomment-2558172424, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGFTJFUPGTMPCZKLSYASEQD2GWMEBAVCNFSM6AAAAABSDNEWQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJYGE3TENBSGQ . You are receiving this because you were mentioned.Message ID: @.***>

Excited to get this done 🚀

[rutikthakre]

@Raxephion Is there any update?

[rutikthakre]

@louis030195 Is there any update?

i'm going to share how we want this designed next week, no point in working on it before

[louis030195]

yes, you need to make it work like macos here

https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-core/src/operator/platforms/windows.rs

[tribhuwan-kumar]

yes, you need to make it work like macos here

https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-core/src/operator/platforms/windows.rs

could you assign it to me? like you said here https://github.com/mediar-ai/screenpipe/pull/1489#issuecomment-2689654578

[rutikthakre]

yes, you need to make it work like macos here

https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-core/src/operator/platforms/windows.rs

Sure 🚀 and Please assign it to me.

[louis030195]

@tribhuwan-kumar assigned

[rutikthakre]

@louis030195 I was working on it since yesterday

[algora-pbc[bot]]

💡 @tribhuwan-kumar submitted a pull request that claims the bounty. You can visit your bounty board to reward.

[algora-pbc[bot]]

💡 @Rutik7066 submitted a pull request that claims the bounty. You can visit your bounty board to reward.

[louis030195]

increased bounty to $400

[rutikthakre]

@louis030195 Can I give it a shot ?

[algora-pbc[bot]]

🎉🎈 @tribhuwan-kumar has been awarded $400 by Screenpi.pe! 🎈🎊