Unable to use cua Agent UI with Windows OS with provider Windows Sandbox

[RBEI-AshwinRU]

Hello team, I am trying to run cua Agent in UI mode and referred the Quickstart(GUI). When I run the agent, Web UI pops up. When I select OS as "windows" and provider as "winsandbox", I get the error "Error: Image must be in the format". I tried multiple times but no success.

Cmd logs: INFO:computer:Logger set to DEBUG level INFO:computer:Initializing Computer... INFO:computer:Logger set to DEBUG level INFO:computer.vm:Logger set to DEBUG level INFO:computer.interface:Logger set to DEBUG level Traceback (most recent call last): File "C:\Users\ashwi\AppData\Local\Programs\Python\Python313\Lib\site-packages\agent\ui\gradio\ui_components.py", line 582, in process_response global_agent = create_agent( model_string=model_string, ...<8 lines>... max_trajectory_budget=max_budget_value if max_budget_value and max_budget_value > 0 else None, ) File "C:\Users\ashwi\AppData\Local\Programs\Python\Python313\Lib\site-packages\agent\ui\gradio\app.py", line 215, in create_agent computer = create_computer_instance( verbosity=verbosity, ...<3 lines>... api_key=computer_api_key ) File "C:\Users\ashwi\AppData\Local\Programs\Python\Python313\Lib\site-packages\agent\ui\gradio\app.py", line 189, in create_computer_instance global_computer = Computer( verbosity=verbosity, ...<3 lines>... api_key=api_key ) File "C:\Users\ashwi\AppData\Local\Programs\Python\Python313\Lib\site-packages\computer\computer.py", line 158, in init raise ValueError("Image must be in the format <image_name>:") ValueError: Image must be in the format <image_name>:

[momadvisor]

I experienced the same error. Even if I directly modify the code to fix the issue, another new problems occur. It seems that Windows support is still not running smoothly. For reference, I tested this on Windows 10.

[momadvisor]

#!/usr/bin/env python3

import asyncio
import logging
from computer import Computer, VMProviderType
from agent import ComputerAgent

async def main():
    logging.basicConfig(level=logging.DEBUG)
    async with Computer(os_type="windows",
                        provider_type=VMProviderType.WINSANDBOX,
                        verbosity=1,
                        use_host_computer_server=False) as computer:
        agent = ComputerAgent(
            model="openai/computer-use-preview",
            tools=[computer],
            verbosity=1,
            only_n_most_recent_images=2,
        )
        
        # Execute a single task
        task = "analyze what you see"
        print(f"Executing task: {task}")
        messages = [{"role": "user", "content": task}]
        async for result in agent.run(messages):
            for item in result["output"]:
                if item["type"] == "message":
                    print(item["content"][0]["text"])

if __name__ == "__main__":
    asyncio.run(main())

[wbste]

Having similar issues...

1. Can't install pywinsandbox from git directly per the docs, Get an error.
2. If I use the pypi version of it, it installs.
3. Running the above debug script, I an IP error.
4. The VM is indeed booted up, and if I launch a sandbox instance manually it works fine.

...
INFO:computer.providers.winsandbox.provider:Windows Sandbox is not running yet (status: stopped). Waiting...
INFO:computer.providers.winsandbox.provider:Waiting for Windows Sandbox windows-11-vanilla_latest IP address (attempt 2)...
...

I assume this is an issue with the pywinsandbox and (hopefully) the latest version on the repo solves it, but I can't get it to build. Tried both via pip in the docks, and with uv via uv add git+https://github.com/karkason/pywinsandbox

--- stderr
Downloading winsandbox/shell_extension/sandbox.ico (36 KB)
Error downloading object: winsandbox/shell_extension/sandbox.ico (123802b): Smudge error: Error downloading winsandbox/shell_extension/sandbox.ico (123802be6343589410cf0a7edf940dfc8aa7451003e2978f9dcae17cea9eb037): error transferring "123802be6343589410cf0a7edf940dfc8aa7451003e2978f9dcae17cea9eb037": [0] remote missing object 123802be6343589410cf0a7edf940dfc8aa7451003e2978f9dcae17cea9eb037
...
fatal: winsandbox/shell_extension/sandbox.ico: smudge filter lfs failed

[momadvisor]

Having similar issues...

1. Can't install pywinsandbox from git directly per the docs, Get an error.
2. If I use the pypi version of it, it installs.
3. Running the above debug script, I an IP error.
4. The VM is indeed booted up, and if I launch a sandbox instance manually it works fine.

...
INFO:computer.providers.winsandbox.provider:Windows Sandbox is not running yet (status: stopped). Waiting...
INFO:computer.providers.winsandbox.provider:Waiting for Windows Sandbox windows-11-vanilla_latest IP address (attempt 2)...
...

I assume this is an issue with the pywinsandbox and (hopefully) the latest version on the repo solves it, but I can't get it to build. Tried both via pip in the docks, and with uv via uv add git+https://github.com/karkason/pywinsandbox

--- stderr
Downloading winsandbox/shell_extension/sandbox.ico (36 KB)
Error downloading object: winsandbox/shell_extension/sandbox.ico (123802b): Smudge error: Error downloading winsandbox/shell_extension/sandbox.ico (123802be6343589410cf0a7edf940dfc8aa7451003e2978f9dcae17cea9eb037): error transferring "123802be6343589410cf0a7edf940dfc8aa7451003e2978f9dcae17cea9eb037": [0] remote missing object 123802be6343589410cf0a7edf940dfc8aa7451003e2978f9dcae17cea9eb037
...
fatal: winsandbox/shell_extension/sandbox.ico: smudge filter lfs failed

Same here. So I manually downloaded pywinsandbox and then pip install the downloaded local path. You could work around in this way. But in windows 10, even when I launched the VM, It failed becuade pywinsandbox session server did not get launched.

[wbste]

Any updates or help needs for testing?

[jamesmurdza]

@wbste We have released a fix. Could you please try the latest version of cua-computer and let us know if it's working for you?

[wbste]

I take it back. Still doesn't work (from here). Sandbox launches but never connects...

# /// script
# dependencies = [
#   "cua-agent[all]",
#   "cua-computer[all]",
#   "pywinsandbox @ git+https://github.com/karkason/pywinsandbox"
# ]
# ///

# Add $env:GIT_LFS_SKIP_SMUDGE="1" to your PowerShell profile if you have issues with Git LFS

import asyncio

from agent import ComputerAgent
from computer import Computer, VMProviderType


async def test_windows_agent():
    # Create Windows Sandbox computer
    computer = Computer(
        provider_type=VMProviderType.WINSANDBOX,
        os_type="windows",
        memory="4GB",
    )

    # Start the VM (~35s)
    await computer.run()

    # Create agent with your preferred model
    agent = ComputerAgent(model="openai/computer-use-preview", save_trajectory=True, tools=[computer])

    # Give it a task
    async for result in agent.run("Open Calculator and compute 15% tip on $47.50"):
        print(f"Agent action: {result}")

    # Shutdown the VM
    await computer.stop()


asyncio.run(test_windows_agent())

Still end up with FileNotFoundError: Sandbox server didn't startup even though the sandbox launches.

[wbste]

Alright, got it. I was using uv for the project. I.e. uv add, uv init... Due to how the venv get's set up it wasn't copying stuff over correctly it seems. Creating a standard py -m venv venv sort of setup works fine. I think I need to explore uv pip install.. commands a bit more, may fix.