Richard Abrich

Implement UGround

3

### Feature request We would like to incorporate UGround for identifying regions of interest: https://x.com/ysu_nlp/status/1844186560901808328 Inference code here: https://github.com/boyugou/llava_uground/ ### Motivation As an alternative to segmentation / description.

### Role TODO: Identify roles within information security, compliance, billing, and operational teams responsible for managing compliance, auditing, data protection, security monitoring, and procurement tasks. ### Task TODO: Identify repetitive,...

docs(README): add chrome_extension Github link

### Feature request We would like to implement a new replay strategy that directly extracts and generates target element coordinates using https://docs.anthropic.com/en/docs/build-with-claude/computer-use and/or https://github.com/OpenAdaptAI/OpenAdapt/issues/880 and/or https://github.com/OpenAdaptAI/OpenAdapt/issues/894. ### Motivation Robustness, speed

### Feature request We want to implement https://huggingface.co/microsoft/OmniParser in a `ReplayStrategy` (e.g. https://github.com/OpenAdaptAI/OpenAdapt/pull/888) ### Motivation > OmniParser is designed to be able to convert unstructured screenshot image into structured list...

### Feature request User @limaconnect on Discord: > Have you taken a look at agency swarm? It's also open source > I think if you incorporated that into your tool...

### Describe the bug No recordings are displayed in the dashboard: ### To Reproduce ``` poetry shell python -m openadapt.entrypoint ``` Click on "Recordings" tab

### Describe the bug ``` poetry shell python -m openadapt.entrypoint ... 2024-10-25 11:21:52.595 | INFO | openadapt.error_reporting:configure_error_reporting:28 - is_reporting_branch=True DB_URL=sqlite:////Users/abrichr/oa/src/OpenAdapt/openadapt/data/openadapt.db INFO [alembic.runtime.migration] Context impl SQLiteImpl. INFO [alembic.runtime.migration] Will assume non-transactional...

### Feature request How can we incorporate the ideas/code from https://arxiv.org/abs/2406.19263? > Graphical User Interfaces (GUIs) are central to our interaction with digital devices. Recently, growing efforts have been made...

[Bug]: The 'Set-ExecutionPolicy' command was found in the module 'Microsoft.PowerShell.Security', but the module could not be loaded.

3

### Describe the bug from @limaconnect on Discord: ``` Set-ExecutionPolicy : The 'Set-ExecutionPolicy' command was found in the module 'Microsoft.PowerShell.Security', but the module could not be loaded. For more information,...