Fails with "json.decoder.JSONDecodeError" trying to manipulate with screen using Antropic

Open artemu78 opened this issue 1 year ago • 1 comments

Describe the bug

I've run "interpreter --os" to play with anthropic OS control feature and it crashed with "json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)"

Reproduce

I've run "interpreter --os", set a key and asked if it can hear me through the mic

it tried to open settings and failed:

I'll click on the Sound settings to check your microphone settings.

{"action": "mouse_move", "coordinate": [40, 369]}{"action": "left_click"}{"action": "screenshot"}

I notice I'm still in the Notifications settings. Let me try clicking the Sound icon again to access sound settings.

{"action": "key", "text": "Command+Space"}{"action": "type", "text": "Sound"}

Let me click the "Open" button to access Sound settings.

{"action": "mouse_move", "coordinate": [558, 202]}Traceback (most recent call last): File "/opt/anaconda3/bin/interpreter", line 5, in from interpreter.terminal_interface.start_terminal_interface import main File "/opt/anaconda3/lib/python3.12/site-packages/interpreter/init.py", line 54, in run_async_main() File "/opt/anaconda3/lib/python3.12/site-packages/interpreter/computer_use/loop.py", line 528, in run_async_main asyncio.run(main()) File "/opt/anaconda3/lib/python3.12/asyncio/runners.py", line 194, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/opt/anaconda3/lib/python3.12/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/opt/anaconda3/lib/python3.12/site-packages/interpreter/computer_use/loop.py", line 505, in main async for chunk in sampling_loop( File "/opt/anaconda3/lib/python3.12/site-packages/interpreter/computer_use/loop.py", line 186, in sampling_loop current_block.input = json.loads(current_block.partial_json) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/lib/python3.12/json/init.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/lib/python3.12/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/lib/python3.12/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)