pymobiledevice3 developer accessibility crash

[prife]

Test environment

• Host OS version.
• Target device model and iOS version.

    {
        "BuildVersion": "17G68",
        "ConnectionType": "USB",
        "DeviceClass": "iPhone",
        "DeviceName": "\u4f59\u59d7\u7684 iPhone",
        "Identifier": "82d8ccbcd9160681f7fd9d377d8e0dff7c6591a5",
        "ProductType": "iPhone10,3",
        "ProductVersion": "13.6"
    },
    {
        "BuildVersion": "20A362",
        "ConnectionType": "USB",
        "DeviceClass": "iPhone",
        "DeviceName": "iPhone X",
        "Identifier": "8a8358e12e0306cc804f4367d9152fb795e3b561",
        "ProductType": "iPhone10,3",
        "ProductVersion": "16.0"
    },

Describe the bug

capabilities

$ python -m pymobiledevice3 developer accessibility capabilities

Traceback (most recent call last):
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pymobiledevice3/__main__.py", line 94, in <module>
    cli()
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pymobiledevice3/__main__.py", line 56, in cli
    cli_commands()
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pymobiledevice3/cli/developer.py", line 725, in accessibility_capabilities
    print_json(AccessibilityAudit(lockdown).device_capabilities())
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pymobiledevice3/services/accessibilityaudit.py", line 136, in device_capabilities
    return self.recv_response()[0]
TypeError: 'generator' object is not subscriptable

test on two devices:

1. iOS 13.6 on iPhoneX
2. iOS 16.0

list-itmes

$ python -m pymobiledevice3 developer accessibility  list-items 

Traceback (most recent call last):
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pymobiledevice3/__main__.py", line 94, in <module>
    cli()
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pymobiledevice3/__main__.py", line 56, in cli
    cli_commands()
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/wetest/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pymobiledevice3/cli/developer.py", line 791, in accessibility_list_items
    current_item = data[0]
IndexError: list index out of range

To Reproduce

1. make the iOS on SpringBoard(Launcher) app.
2. run pymobiledevice3 developer accessibility list-items

Expected behavior

I've no ieda about the accessibility subcommand

Logs

Additional context

[doronz88]

Regarding:

pymobiledevice3 developer accessibility list-items

Thanks for checking it out. It supposed to list all the UIElements in the current displayed app (as can be done fron Accessibility Manager). From some reason, it seems the event that on previous versions reported the contents of the selected item now doesn't so there's probably some new way to do this. This will require further research to fix.

BTW, the Preferences app for example still works just fine using this same protocol - I managed to list all displayed buttons. I'll update when I'll have a fix for that.

[doronz88]

As for the capabilities issue, it should be fixed in my PR (#406).

[doronz88]

@prife you can already checkout latest release with the bugfixes. The list-items will require further work though.

[prife]

Hi doronz, I found that you had implemented perform_press in services/accessibilityaudit.py. I'm very curious about it, but I don't know how to call it. Could you give me some help?

[doronz88]

Assuming the application in has the get-task-allow entitlement and you already have an AXElement value (the one you might get after listing all of them), then you can simulate a press

[doronz88]

I believe this issue has been resolved with latest releases - please update