Memento Accessibility API

Is this connected to the accessibility API to retrieve context information (like url, app name, etc) yet?

Sep 21 '23 18:09 jlia0

Hi, I tinkered a bit with the accessibility API to try to extract the text directly from the apps instead of using an OCR, but did not achieve much. If you have any good reference material it would be great, or you can make a PR :)

Sep 21 '23 21:09 apirrone

Here's one that cyte2 was referencing from: https://github.com/tmandry/AXSwift

However I am not sure if there is a Python API for it, do you mind sharing some of your tinkering code?

I believe we still need OCR for extracting the text, the accessibility api is for extracting "metadata" like url or window contexts.

Sep 21 '23 22:09 jlia0

Unfortunately I don't seem to have kept my tinkering code :/ I tried to use ORCA screen reader (https://github.com/GNOME/orca) but it was not the right tool I think

Sep 23 '23 08:09 apirrone

https://kevinchen.co/blog/rewind-ai-app-teardown/

^^^ I think this would probably help

Oct 09 '23 01:10 jlia0

Yes this blog post was very helpful :)

Oct 09 '23 08:10 apirrone