TagUI
TagUI copied to clipboard
aisingapore
AI Recorder to record web and desktop workflows to various formats - to explore
Hi @ruthtxh I'm considering closing this for now since I don't have the bandwidth to add this now. If you are adding, the gist is creating a desktop recorder and automate creating the images for visual automation from user's clicks and keystrokes.
Ie need some image process logic (or ML of common UI elements as fallback) to know how far from the click point to crop the screen to use as the element image identifier. I believe it is technical possible to create a usable recorder.
The main downside of such an approach is ambiguity of recording, but users can always fine-tune by snapping on their own the incorrect generated snapshots. If done well, can work for both desktop and web applications.
Maybe I'll assign to you to review with PM if this is worth exploring.
The various formats in the issue title mean exporting to TagUI human language .tag, .docx, .py (RPA for Python), .cs (TagUI for C#) etc. But the main benefit is for the human language users. The technical users might not require this recorder.