Background
System: Windows 11
Application: Microsoft PC Manager
Code: Main branch of UFO. https://github.com/microsoft/UFO/tree/main
LLM: gpt-4o - Azure OpenAI (AOAI)
Task Instructions
1.start the MSPCManager desktop application;
2.click the 'Settings' button;
3.select 'Dark' for appearance.
Expected Process
1.start the MSPCManager desktop application;
2.click the 'Settings' button;
3.select 'Dark' for appearance.
4.Shown last
Real Process
1.start the MSPCManager desktop application;
2.click the 'Settings' button;
3.select 'Dark' for appearance.
4.Then an error occurred.
Error Occurs at execute_action
Traceback (most recent call last):
File "E:\WicreSoft\Tasks\Codes\GUIAutomationTest\UFO\ufo\agents\processors\basic.py", line 192, in wrapper
func(self, *args, **kwargs)
File "E:\WicreSoft\Tasks\Codes\GUIAutomationTest\UFO\ufo\agents\processors\basic.py", line 174, in wrapper
result = func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\WicreSoft\Tasks\Codes\GUIAutomationTest\UFO\ufo\agents\processors\app_agent_processor.py", line 558, in execute_action
self.actions.execute_all(
File "E:\WicreSoft\Tasks\Codes\GUIAutomationTest\UFO\ufo\agents\processors\actions.py", line 479, in execute_all
action.action_flow(puppeteer, control_dict, application_window)
File "E:\WicreSoft\Tasks\Codes\GUIAutomationTest\UFO\ufo\agents\processors\actions.py", line 270, in action_flow
puppeteer.receiver_manager.create_ui_control_receiver(
File "E:\WicreSoft\Tasks\Codes\GUIAutomationTest\UFO\ufo\automator\puppeteer.py", line 202, in create_ui_control_receiver
self.ui_control_receiver = factory.create_receiver(control, application)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\WicreSoft\Tasks\Codes\GUIAutomationTest\UFO\ufo\automator\ui_control\controller.py", line 442, in create_receiver
return ControlReceiver(control, application)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\WicreSoft\Tasks\Codes\GUIAutomationTest\UFO\ufo\automator\ui_control\controller.py", line 50, in init
self.wait_enabled()
File "E:\WicreSoft\Tasks\Codes\GUIAutomationTest\UFO\ufo\automator\ui_control\controller.py", line 346, in wait_enabled
while not self.control.is_enabled():
^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\WicreSoft\Tasks\Codes\GUIAutomationTest\UFO.venv\Lib\site-packages\pywinauto\base_wrapper.py", line 344, in is_enabled
return self.element_info.enabled #and self.top_level_parent().element_info.enabled
^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\WicreSoft\Tasks\Codes\GUIAutomationTest\UFO.venv\Lib\site-packages\pywinauto\uia_element_info.py", line 321, in enabled
return bool(self._element.CurrentIsEnabled)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ctypes.COMError: (-2147220991, 'An event was unable to invoke any of the subscribers', (None, None, None, 0, None))
Response:
{"Observation": "The current screenshot shows the PC Manager application with the 'Settings' tab open. The 'Appearance' ComboBox is visible with options 'System', 'Light', and 'Dark'. The last action of clicking 'Appearance' took effect, as the dropdown menu is open.", "Thought": "The sub-task is to click the 'Settings' button in the MSPCManager application. However, the 'Settings' tab is already open, as indicated by the highlighted 'Settings' tab on the left sidebar. Therefore, I should proceed to the next step, which is to select 'Dark' from the 'Appearance' dropdown menu.", "ControlLabel": "19", "ControlText": "Dark", "Function": "click_input", "Args": {"button": "left", "double": false}, "Status": "CONTINUE", "Plan": ["<FINISH>"], "Comment": "The 'Settings' tab is already open, so I proceeded to select 'Dark' from the 'Appearance' dropdown menu to complete the user request.", "SaveScreenshot": {"save": false, "reason": ""}}
5.Shown last
The section of the report related to the cause of the error
Logs
evaluation.log
request.log
response.log
microsoft
