OpenAdapt

OpenAdapt copied to clipboard

Consolidate video.py and capture.py for local hardware acceleration

5

### Feature request `capture/_macos.py` uses `AVFoundation`, `capture/_windows.py ` uses `screen_recorder_sdk` which uses `MediaFoundationAPI`. These are likely to be more performant than `mss` used in `record.py` and `video.py`, but currently `capture`...

abrichr

Github Action to build via `openadapt.app.build`

4

### Feature request We wish to build MacOS and Windows binaries and have them stored to https://github.com/OpenAdaptAI/OpenAdapt/releases. ### Motivation We want to add a "Download App" button to www.openadapt.ai

abrichr

Implement client/server functionality

5

### Feature request To support https://github.com/openadaptai/SoM we need to implement a client.py with https://www.gradio.app/docs/client. See: - client.py: https://github.com/microsoft/SoM/pull/19/files#diff-1ebfaf6cb3592166b73835fa82333cb7109e7c624865c0039a7b22ff34aa27fa) - deploy.py: https://github.com/microsoft/SoM/pull/19/files#diff-5c9ed18af9a5f902219d12c3044ccb193c2c304a3748d02702889c2ca5703978 ### Motivation https://github.com/openadaptai/SoM is state-of-the-art for visual understanding, and...

abrichr

### Feature request We wish to implement https://wow.groq.com/ See: - [`openadapt.adapters.openai`](https://github.com/OpenAdaptAI/OpenAdapt/pull/610/files#diff-8a5ff1c90e18e493d2077b9298882ab872052c285ba5b58bc1f40ea7983740af) - [`openadapt.adapters.anthropic`](https://github.com/OpenAdaptAI/OpenAdapt/pull/610/files#diff-9c007eedecc63488849c0f14040031db3e938e7b03a34c3b0adf0291163211e5) ### Motivation Fast inference (thanks @FFFiend!)

abrichr

### Feature request https://arxiv.org/abs/2211.11559 > We present VISPROG, a neuro-symbolic approach to solving complex and compositional visual tasks given natural language instructions. VISPROG avoids the need for any task-specific training....

abrichr

Deploy PromptFlow to AWS

4

### Feature request We wish to automate the deployment of https://microsoft.github.io/promptflow/how-to-guides/tune-prompts-with-variants.html to AWS. e.g. under `openadapt/server/promptflow.py` (based on https://github.com/OpenAdaptAI/SoM/blob/main/deploy.py): ``` import os import subprocess import boto3 from loguru import logger...

abrichr

### Describe the bug We expect the user's system to be relatively unaffected by the capture process. When capturing on a Windows device the mouse consistently flickers, even when resource...

gawnecoding

### Feature request We wish to export a `recording: openadapt.models.Recording`' to PowerAutomate code. This will likely use the Process Graph in https://github.com/OpenAdaptAI/OpenAdapt/pull/560 as input. ### Motivation _No response_

abrichr

### Feature request We wish to implement a Process API by prompting an LLM with the Process Graph and/or Screenshots/Actions, e.g. with Chain-of-Code   ### Motivation LLMs are great...

abrichr

### Feature request We would like to implement a `config.REPLAY_ENABLE_ACTION_CONFIRMATION: bool` that prompts the user via a desktop dialog (e.g. transparent modal, system tray button) that describes to the user...

abrichr