Set-of-Mark Prompting (via Segment-Anything-Model) + GPT4-V

[abrichr]

Work-in-progress.

Related: https://github.com/OpenAdaptAI/SoM

sam_mixin.py:

Scratch:

Via https://huggingface.co/spaces/Roboflow/SoM :

gpt4v_coords.py:

gpt4v_grid.py:

Addresses https://github.com/OpenAdaptAI/OpenAdapt/issues/513

[sweep-ai-deprecated[bot]]

Apply Sweep Rules to your PR?

• [ ] Apply: All new business logic should have corresponding unit tests.
• [ ] Apply: Refactor large functions to be more modular.
• [ ] Apply: Add docstrings to all functions and file headers.

[abrichr]

Possibly relevant: https://github.com/ksugar/samapi