UI-TARS-desktop icon indicating copy to clipboard operation
UI-TARS-desktop copied to clipboard

[Bug Report]: SDK bug __WEBPACK_EXTERNAL_MODULE__ui_tars_action_parser_e6c10e92__.actionParser

Open learningpro opened this issue 7 months ago • 3 comments

Version

1.2.0

Issue Type

  • [x] Select a issue type 👇
  • [x] Agent Core
  • [ ] MCP
  • [ ] Search
  • [ ] Browser
  • [ ] Setting
  • [ ] UI
  • [ ] Other (please specify in description)

Model Provider

  • [ ] Select a model provider 👇
  • [ ] Anthropic
  • [ ] OpenAI
  • [ ] Azure OpenAI
  • [x] Other (please specify in description)

Search Provider

  • [ ] Select a model provider 👇
  • [ ] Local Browser Search
  • [ ] Travily
  • [ ] Duckduckgo
  • [ ] SearXNG
  • [x] Other (please specify in description)

Problem Description

模型使用的是火山引擎的 doubao-1.5-ui-tars-250328 参考 https://github.com/bytedance/UI-TARS-desktop/blob/main/docs/sdk.md 的样例,运行时候报错,鼠标没有移动也没有点击、键入等操作,一动不动

[UITarsModel] invoke: screenContext={"width":1920,"height":1080}, scaleFactor=2, uiTarsVersion=undefined [UITarsModel cost]: 3382ms [UITarsModel] error TypeError: (0 , WEBPACK_EXTERNAL_MODULE__ui_tars_action_parser_e6c10e92.actionParser) is not a function at UITarsModel.invoke (file:///Users/apple/Documents/computeruse/node_modules/@ui-tars/sdk/dist/Model.mjs:85:134) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async retries._retry_model (file:///Users/apple/Documents/computeruse/node_modules/@ui-tars/sdk/dist/GUIAgent.mjs:167:40) at async GUIAgent.run (file:///Users/apple/Documents/computeruse/node_modules/@ui-tars/sdk/dist/GUIAgent.mjs:165:59) at async file:///Users/apple/Documents/computeruse/index.js:23:5 [GUIAgent Response]: Thought: 我看了看当前的情况,地址栏里的内容还在那里,看来之前的点击操作并没有成功选中它。不过没关系,我知道该怎么做 - 直接点击地址栏,

但是运行 npx @ui-tars/cli start 的时候就能正确执行,没有报错

电脑是MacOS M2 15.4版本的系统

Error Logs

Action: hotkey(key='ctrl a') GUIAgent Parsed Predictions: [] received data [GUIAgent] loopCnt: 2 [NutjsOperator] scaleX 2 scaleY 2 [NutjsOperator] screenshot: 1920x1080, scaleFactor: 2 received data [UITarsModel] invoke: screenContext={"width":1920,"height":1080}, scaleFactor=2, uiTarsVersion=undefined [UITarsModel cost]: 2838ms [UITarsModel] error TypeError: (0 , WEBPACK_EXTERNAL_MODULE__ui_tars_action_parser_e6c10e92.actionParser) is not a function at UITarsModel.invoke (file:///Users/apple/Documents/computeruse/node_modules/@ui-tars/sdk/dist/Model.mjs:85:134) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async retries._retry_model (file:///Users/apple/Documents/computeruse/node_modules/@ui-tars/sdk/dist/GUIAgent.mjs:167:40) at async GUIAgent.run (file:///Users/apple/Documents/computeruse/node_modules/@ui-tars/sdk/dist/GUIAgent.mjs:165:59) at async file:///Users/apple/Documents/computeruse/index.js:23:5 [GUIAgent Response]: Thought: 我发现刚才的操作有点问题,地址栏里的内容还在那里。看来我需要重新点击一下地址栏,把当前这个npmjs的网址选中,这样才能继续下一步输入bilibili的网址。 Action: click(start_box='163 82 163 82') GUIAgent Parsed Predictions: [] received data [GUIAgent] loopCnt: 3 [NutjsOperator] scaleX 2 scaleY 2 [NutjsOperator] screenshot: 1920x1080, scaleFactor: 2 received data [UITarsModel] invoke: screenContext={"width":1920,"height":1080}, scaleFactor=2, uiTarsVersion=undefined [UITarsModel cost]: 3382ms [UITarsModel] error TypeError: (0 , WEBPACK_EXTERNAL_MODULE__ui_tars_action_parser_e6c10e92.actionParser) is not a function at UITarsModel.invoke (file:///Users/apple/Documents/computeruse/node_modules/@ui-tars/sdk/dist/Model.mjs:85:134) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async retries._retry_model (file:///Users/apple/Documents/computeruse/node_modules/@ui-tars/sdk/dist/GUIAgent.mjs:167:40) at async GUIAgent.run (file:///Users/apple/Documents/computeruse/node_modules/@ui-tars/sdk/dist/GUIAgent.mjs:165:59) at async file:///Users/apple/Documents/computeruse/index.js:23:5 [GUIAgent Response]: Thought: 我看了看当前的情况,地址栏里的内容还在那里,看来之前的点击操作并没有成功选中它。不过没关系,我知道该怎么做 - 直接点击地址栏,把这个npmjs的网址选中,这样就能为输入bilibili的网址做好准备了。 Action: click(start_box='163 82 163 82') GUIAgent Parsed Predictions: [] received data [GUIAgent] loopCnt: 4 [NutjsOperator] scaleX 2 scaleY 2 [NutjsOperator] screenshot: 1920x1080, scaleFactor: 2 received data

learningpro avatar Apr 30 '25 10:04 learningpro

Update the action parser package and recompile it. Looks like there might be a packaging issue with the action parser.

ZhaoHeh avatar Apr 30 '25 13:04 ZhaoHeh

Update the action parser package and recompile it. Looks like there might be a packaging issue with the action parser.

Is this problem solved?

Lee0624 avatar May 26 '25 07:05 Lee0624

i'm having the same issue too

refself avatar May 27 '25 15:05 refself