cua icon indicating copy to clipboard operation
cua copied to clipboard

[BUG] Unsupported provider type: winsandbox

Open Katehuuh opened this issue 6 months ago • 7 comments

Since pywinsandbox example works fine, I tried winsandbox_example.py with log:

(venv) C:\>winsandbox_example.py
Starting Windows Sandbox...
Failed to initialize provider context: Unsupported provider type: winsandbox
Failed to initialize computer: Failed to initialize VM provider: Unsupported provider type: winsandbox
Error: Failed to initialize computer: Failed to initialize VM provider: Unsupported provider type: winsandbox
Traceback (most recent call last):
  File "C:\venv\lib\site-packages\computer\computer.py", line 266, in run
    raise ValueError(f"Unsupported provider type: {self.provider_type}")
ValueError: Unsupported provider type: winsandbox

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\venv\lib\site-packages\computer\computer.py", line 280, in run
    raise RuntimeError(f"Failed to initialize VM provider: {e}")
RuntimeError: Failed to initialize VM provider: Unsupported provider type: winsandbox

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\winsandbox_example.py", line 22, in main
    await computer.run()
  File "C:\venv\lib\site-packages\computer\computer.py", line 385, in run
    raise RuntimeError(f"Failed to initialize computer: {e}")
RuntimeError: Failed to initialize computer: Failed to initialize VM provider: Unsupported provider type: winsandbox
Stopping Windows Sandbox...
Windows Sandbox stopped.

I want to run cua locally, so I'm hoping to use winsandbox + UITARS-1.5 (via VLM/transformer on Windows 11, torch 2.7.1+cu128, Python 3.10.8). (Plz, support Python 3.10 by avoiding or wrapping from enum import StrEnum. Also, let cua run without a virtual sandbox... #118)

Katehuuh avatar Jun 24 '25 16:06 Katehuuh

To run the winsandbox_example.py on Python 3.10, you'll have to change the start of the example to:

import asyncio
from computer import Computer, VMProviderType

async def main():
    """Test the Windows Sandbox provider."""
    
    # Create a computer instance using Windows Sandbox
    computer = Computer(
        provider_type=VMProviderType.WINSANDBOX,  # Use enum instead of string
        os_type="windows",
        memory="4GB",
        # ephemeral=True,  # Always true for Windows Sandbox
    )
    
    # Rest of the example remains the same...

The key change is importing VMProviderType and using VMProviderType.WINSANDBOX instead of the string "winsandbox". If you want to run cua locally on your own computer without Windows Sandbox, we support that too:

# Install and start computer-server
pip install cua-computer-server
python -m computer_server

Then in your code:

computer = Computer(os_type="windows", use_host_computer_server=True)

This bypasses any VM layer and runs c/ua directly on your host machine.

ddupont808 avatar Jun 24 '25 19:06 ddupont808

(venv) C:\>python -m computer_server
INFO:computer_server.cli:Starting CUA Computer API server on 0.0.0.0:8000...
INFO:computer_server.cli:HTTP mode (no SSL certificates provided)
INFO:     Started server process [17004]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     127.0.0.1:61329 - "GET / HTTP/1.1" 404 Not Found
INFO:     127.0.0.1:61329 - "GET /favicon.ico HTTP/1.1" 404 Not Found

Second new CMD with script modification:

+ from computer import Computer, VMProviderType
...
+ VMProviderType.WINSANDBOX

log

Traceback (most recent call last):
  File "C:\winsandbox_example.py", line 47, in <module>
    asyncio.run(main())
  File "C:\Python\Python310\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "C:\Python\Python310\lib\asyncio\base_events.py", line 649, in run_until_complete
    return future.result()
  File "C:\winsandbox_example.py", line 10, in main
    provider_type=VMProviderType.WINSANDBOX,  # Use enum instead of string
  File "C:\Python\Python310\lib\enum.py", line 436, in __getattr__
    raise AttributeError(name) from None
AttributeError: WINSANDBOX

As for computer = Computer(os_type="windows", use_host_computer_server=True)

Starting Windows Sandbox...
Failed to initialize computer: Unsupported OS type: windows
Error: Failed to initialize computer: Unsupported OS type: windows
Traceback (most recent call last):
  File "C:\venv\lib\site-packages\computer\computer.py", line 205, in run
    InterfaceFactory.create_interface_for_os(
  File "C:\venv\lib\site-packages\computer\interface\factory.py", line 39, in create_interface_for_os
    raise ValueError(f"Unsupported OS type: {os}")
ValueError: Unsupported OS type: windows

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\winsandbox_example.py", line 13, in main
    await computer.run()
  File "C:\venv\lib\site-packages\computer\computer.py", line 385, in run
    raise RuntimeError(f"Failed to initialize computer: {e}")
RuntimeError: Failed to initialize computer: Unsupported OS type: windows
Stopping Windows Sandbox...
Windows Sandbox stopped.

Katehuuh avatar Jun 24 '25 19:06 Katehuuh

Could you verify you're using the latest cua-computer and cua-agent packages with pip show cua-computer cua-agent? Windows support wasn't added until recently, the latest versions are cua-computer==0.2.13 cua-agent==0.2.14

ddupont808 avatar Jun 24 '25 21:06 ddupont808

No, i've cua-computer 0.2.6 and cua-agent 0.2.5 wich was the lastest version valid for py3.10. Now, if upgrading with pip install cua-computer cua-agent --ignore-requires-python --upgrade, it leads me to version cua-computer==0.2.13 cua-agent==0.2.14.

Now winsandbox_example.py Include the modification, log:

Starting Windows Sandbox...
Failed to create Windows Sandbox macos-sequoia-cua_latest: new_sandbox() got an unexpected keyword argument 'memory_mb'
Stack trace: Traceback (most recent call last):
  File "C:\venv\lib\site-packages\computer\providers\winsandbox\provider.py", line 265, in run_vm
    sandbox = winsandbox.new_sandbox(
TypeError: new_sandbox() got an unexpected keyword argument 'memory_mb'

Now, for..

computer = Computer(os_type="windows", use_host_computer_server=True)

This bypasses any VM layer and runs c/ua directly on your host machine.

This seems more promising:

Starting Windows Sandbox...
Windows Sandbox is ready!
IP Address: 127.0.0.1
Testing basic functionality...
Screenshot taken: 1440121 bytes
Testing command execution...
Command output: Hello from Windows Sandbox!

Press any key to continue...

Stopping Windows Sandbox...
Windows Sandbox stopped.

and

(venv) C:\>python -m computer_server
INFO:computer_server.cli:Starting CUA Computer API server on 0.0.0.0:8000...
INFO:computer_server.cli:HTTP mode (no SSL certificates provided)
INFO:     Started server process [32860]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     127.0.0.1:62048 - "WebSocket /ws" [accepted]
INFO:     connection open
INFO:     127.0.0.1:62050 - "WebSocket /ws" [accepted]
INFO:     connection open
INFO:computer_server.main:Client disconnected
INFO:     connection closed
INFO:computer_server.main:Client disconnected
INFO:     connection closed

Katehuuh avatar Jun 24 '25 22:06 Katehuuh

The memory_mb argument was added in a recent version of pywinsandbox that hasn't been released to PyPI yet. You'll need to install from the GitHub repo:

pip install -U git+git://github.com/karkason/pywinsandbox.git

That said, I agree that adding compatibility for older python & pywinsandbox versions in the meantime would make this easier for users.

ddupont808 avatar Jun 25 '25 12:06 ddupont808

  • With pip install -U git+https://github.com/karkason/pywinsandbox.git which upgrade from 1.4.0 to pywinsandbox==1.4.1
Starting Windows Sandbox...
C:\Users\WDAGUtilityAccount\Desktop\venv\Scripts\python.exe C:\Users\WDAGUtilityAccount\Desktop\venv\lib\site-packages\winsandbox\target.py --disable-firewall C:\Users\WDAGUtilityAccount\Desktop\shared_windows_sandbox_dir\server_address C:\Users\WDAGUtilityAccount\Desktop\site-packages
Failed to create Windows Sandbox macos-sequoia-cua_latest: Sandbox server didn't startup
Stack trace: Traceback (most recent call last):
  File "C:\venv\lib\site-packages\computer\providers\winsandbox\provider.py", line 265, in run_vm
    sandbox = winsandbox.new_sandbox(
  File "C:\venv\lib\site-packages\winsandbox\launch.py", line 35, in new_sandbox
    return OnlineSandbox(config)
  File "C:\venv\lib\site-packages\winsandbox\sandbox.py", line 41, in __init__
    self._connection_tuple = session.connect_to_sandbox()
  File "C:\venv\lib\site-packages\winsandbox\session\online_session.py", line 137, in connect_to_sandbox
    raise FileNotFoundError("Sandbox server didn't startup")
FileNotFoundError: Sandbox server didn't startup

However, Sandbox windows do open but no interaction occurs; Error traceback also takes half a min to appear for example pywinsandbox.

  • Related: https://github.com/karkason/pywinsandbox/issues/14

    Edit1: The Error: FileNotFoundError: Sandbox server didn't startup is /venv Issue with PR https://github.com/karkason/pywinsandbox/pull/15 haven't been able to fix, Because without a Python virtual environment, the example pywinsandbox does work in regular (non-venv), but it is bad practice!

Katehuuh avatar Jun 25 '25 14:06 Katehuuh

lt's recap:

  1. Fix for Python 3.10: pip install cua-computer cua-agent --ignore-requires-python --upgrade + edit winsandbox_example.py:
+ from computer import Computer, VMProviderType
...
+        provider_type=VMProviderType.WINSANDBOX,
  1. pip install -U git+https://github.com/karkason/pywinsandbox.git Fixed unexpected keyword argument 'memory_mb'

  1. computer = Computer(os_type="windows", use_host_computer_server=True) + python -m computer_server use natively win.

Unresolved:

  1. pywinsandbox fail Error: FileNotFoundError: Sandbox server didn't startup, root cause target.py fails verification import?
  2. Unavailable on Windows: UITARS-1.5 requires Apple MLX, and vLLM got common compilation issue: ModuleNotFoundError: No module named 'vllm._C'; Maybe simply use transfomer? "AgentLoop.UITARS_PYTORCH"
  3. Native Windows: manual operations work without agent; However, not sure if gemini OpenAI-compatible endpoint is supported:
"""CUA Windows Test - Gemini + OMNI Loop; Required `set GEMINI_API_KEY=ur-API` + separate CMD: `python -m computer_server`"""
import asyncio, os
from computer import Computer, VMProviderType
from agent import ComputerAgent, LLM, AgentLoop, LLMProvider

async def test_cua():
    os.environ["OPENAI_API_KEY"] = os.environ.get("GEMINI_API_KEY", "")
    
    computer = Computer(os_type="windows", use_host_computer_server=True) # Alternative using Windows Sandbox: Computer(provider_type=VMProviderType.WINSANDBOX, os_type="windows")
    await computer.run()
    # await computer.interface.right_click(100, 100) # Alternative manual operations: get_screen_size(),  get_cursor_position(), screenshot(), drag_to(200,200), hotkey("win","r"), type_text("text"), set_clipboard("test")
    
    agent = ComputerAgent(
        computer=computer,
        loop=AgentLoop.OMNI, # Alternative: .ANTHROPIC, .UITARS, .OMNI (SoM) 
        model=LLM(
            provider=LLMProvider.OAICOMPAT, # OPENAI,ANTHROPIC,OLLAMA,MLXVLM,OAICOMPAT
            name="gemini-2.5-flash",
            provider_base_url="https://generativelanguage.googleapis.com/v1beta/openai/"
        )
    )

    try:
        async for result in agent.run("Open Calculator and compute 25 * 4"):
            print(f"Agent: {result.get('text', '')}")
    except Exception as e:
        print(f"❌ Error: {e}")
    
    await computer.disconnect()

if __name__ == "__main__":
    asyncio.run(test_cua())

log

OCR failed: module 'signal' has no attribute 'SIGALRM'
Error in API call: 'list' object has no attribute 'get'
Unexpected error in API call: 'list' object has no attribute 'get'

Katehuuh avatar Jun 27 '25 15:06 Katehuuh