alibaba
tidevice 执行自动化的时候可以支持iPhone手机录屏功能吗
需要在执行自动化过程中能够录屏手机的操作过程 @zy02636 @codeskyblue @alibaba-oss @373743261 @pengchenglin
看Testerhome上有个帖子，实现了一套录屏的方案 链接：https://github.com/YueChen-C/ios-screen-record
看Testerhome上有个帖子，实现了一套录屏的方案 链接：https://github.com/YueChen-C/ios-screen-record
看这个环境是要在Mac电脑上，但是我用你们的框架，是Windows接着iPhone，能支持吗 @codeskyblue
我写了一个例子，使用imageio结合WDA的mjpeg server录的屏幕，你可以参考一下（因为是通过图片合成的视频，理论上是跨平台的 https://github.com/alibaba/taobao-iphone-device/blob/d6071b6236976dcfd469a5175b019a9aeef209f6/scripts/uitest_screenrecord.py
我写了一个例子，使用imageio结合WDA的mjpeg server录的屏幕，你可以参考一下（因为是通过图片合成的视频，理论上是跨平台的 https://github.com/alibaba/taobao-iphone-device/blob/d6071b6236976dcfd469a5175b019a9aeef209f6/scripts/uitest_screenrecord.py 好的，我试下，3q
我写了一个例子，使用imageio结合WDA的mjpeg server录的屏幕，你可以参考一下（因为是通过图片合成的视频，理论上是跨平台的 https://github.com/alibaba/taobao-iphone-device/blob/d6071b6236976dcfd469a5175b019a9aeef209f6/scripts/uitest_screenrecord.py 好的，我试下，3q
执行到这句：
Read image from WDA mjpeg server
pconn = t.create_inner_connection(9100) # default WDA mjpeg server port
sock = pconn.get_socket()
buf = SocketBuffer(sock)
会提示： sock = pconn.get_socket() AttributeError: 'PlistSocket' object has no attribute 'get_socket' 请问怎么解决 @codeskyblue
我写了一个例子，使用imageio结合WDA的mjpeg server录的屏幕，你可以参考一下（因为是通过图片合成的视频，理论上是跨平台的 https://github.com/alibaba/taobao-iphone-device/blob/d6071b6236976dcfd469a5175b019a9aeef209f6/scripts/uitest_screenrecord.py
请问这个怎么用呢，执行 ：py.test -v scripts/uitest_screenrecord.py -s 后就卡在命令行了，是要自己写什么方法吗
这个好像不支持录制多个视频，因为WDA mjpeg server目前是默认的9100，多个视频设运行时，后面执行的时候会抢占该端口的数据，这个要怎么解决 ，或者说怎么修改mjpeg server的默认端口？求助，急！！@codeskyblue
对了，包括8100也是，每个设备自己设备内监听，不冲突
Exception in thread Thread-31: Traceback (most recent call last): File "D:\Users\yl1150\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner self.run() File "D:\Users\yl1150\AppData\Local\Programs\Python\Python38\lib\threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "F:\GitLab\apprunner\apprunner\utils\wda_record_helper.py", line 65, in _drain self._writer.append_data(im) File "F:\GitLab\apprunner\venv\lib\site-packages\imageio\core\format.py", line 502, in append_data return self._append_data(im, total_meta) File "F:\GitLab\apprunner\venv\lib\site-packages\imageio\plugins\ffmpeg.py", line 574, in _append_data self._write_gen.send(im) ValueError: generator already executing [swscaler @ 0000018d7fc40f80] Warning: data is not aligned! This can lead to a speed loss @codeskyblue 这个报错是因为什么呢，后一个设备启动起来开始录屏，会报这个错误
执行这一步：
@pytest.fixture
def c() -> wda.Client:
_c = wda.USBClient()
_c.unlock()
yield _c
会报错：
self.wda_client = wda.USBClient()
File "F:\GitLab\apprunner\venv\lib\site-packages\wda_init.py", line 1768, in init
raise RuntimeError("more then one device connected")
RuntimeError: more then one device connected
多设备貌似这一步不支持，看了下源码：
class USBClient(Client):
""" connect device through unix:/var/run/usbmuxd """
def __init__(self, udid: str = "", port: int = 8100, wda_bundle_id=None):
if not udid:
usbmux = Usbmux()
infos = usbmux.device_list()
if len(infos) == 0:
raise RuntimeError("no device connected")
elif len(infos) >= 2:
raise RuntimeError("more then one device connected")
udid = infos[0]['SerialNumber']
super().__init__(url=requests_usbmux.DEFAULT_SCHEME + "{}:{}".format(udid, port))
if self.is_ready():
return
_start_wda_xctest(udid, wda_bundle_id)
if not self.wait_ready(timeout=20):
raise RuntimeError("wda xctest launched but check failed")
这里不支持连接两个以上的手机吗，这个是怎么回事，还有我不小心把这个issus给close了，你那边能重新开启吗？ @codeskyblue
大神你好。create_inner_connection这个函数传入的本来应该默认是9100，但是我设备接入stf了，所以9100已经被转发到本地了，那这个里面的端口应该填什么呢？因为我看tidevice是自己通过uuid获取的ip地址…
@codeskyblue 大神你好。create_inner_connection这个函数传入的本来应该默认是9100，但是我设备接入stf了，所以9100已经被转发到本地了，那这个里面的端口应该填什么呢？因为我看tidevice是自己通过uuid获取的ip地址…
/Users/gaojs/.pyenv/versions/3.10.1/bin/python3.10 "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py" --target record_iOS.py::test_main Testing started at 11:19 ... Launching pytest with arguments record_iOS.py::test_main --no-header --no-summary -q in /Users/gaojs/PycharmProjects/airtest-dcck-performance/util
============================= test session starts ============================== collecting ... collected 1 item
record_iOS.py::test_main FAILED [100%]screenrecord is ready to begin
record_iOS.py:107 (test_main) c = <wda.USBClient object at 0x10d2aa890>
def test_main(c: wda.Client):
t = tidevice.Device('00008030-0016685A3AA3C02E')
with make_screenrecord(c, t, "ggg.mp4"):
record_iOS.py:110:
../../../.pyenv/versions/3.10.1/lib/python3.10/contextlib.py:135: in enter return next(self.gen) record_iOS.py:77: in make_screenrecord wr = imageio.get_writer(output_video_path, fps=_fps) ../../../.pyenv/versions/3.10.1/lib/python3.10/site-packages/imageio/v2.py:189: in get_writer image_file = imopen(uri, "w" + mode, **imopen_args)
uri = 'ggg.mp4', io_mode = 'w?', plugin = None, extension = None format_hint = None, legacy_mode = True, kwargs = {} request = <imageio.core.request.Request object at 0x10d277c10> source = 'ggg.mp4'
def imopen(
uri,
io_mode,
*,
plugin=None,
extension=None,
format_hint=None,
legacy_mode=False,
**kwargs,
):
"""Open an ImageResource.
.. warning::
This warning is for pypy users. If you are not using a context manager,
remember to deconstruct the returned plugin to avoid leaking the file
handle to an unclosed file.
Parameters
----------
uri : str or pathlib.Path or bytes or file or Request
The :doc:`ImageResource <../../user_guide/requests>` to load the
image from.
io_mode : str
The mode in which the file is opened. Possible values are::
``r`` - open the file for reading
``w`` - open the file for writing
Depreciated since v2.9:
A second character can be added to give the reader a hint on what
the user expects. This will be ignored by new plugins and will
only have an effect on legacy plugins. Possible values are::
``i`` for a single image,
``I`` for multiple images,
``v`` for a single volume,
``V`` for multiple volumes,
``?`` for don't care (default)
plugin : str, Plugin, or None
The plugin to use. If set to None (default) imopen will perform a
search for a matching plugin. If not None, this takes priority over
the provided format hint.
extension : str
If not None, treat the provided ImageResource as if it had the given
extension. This affects the order in which backends are considered, and
when writing this may also influence the format used when encoding.
format_hint : str
Deprecated. Use `extension` instead.
legacy_mode : bool
If true (default) use the v2 behavior when searching for a suitable
plugin. This will ignore v3 plugins and will check ``plugin``
against known extensions if no plugin with the given name can be found.
**kwargs : Any
Additional keyword arguments will be passed to the plugin upon
construction.
Notes
-----
Registered plugins are controlled via the ``known_plugins`` dict in
``imageio.config``.
Passing a ``Request`` as the uri is only supported if ``legacy_mode``
is ``True``. In this case ``io_mode`` is ignored.
Using the kwarg ``format_hint`` does not enforce the given format. It merely
provides a `hint` to the selection process and plugin. The selection
processes uses this hint for optimization; however, a plugin's decision how
to read a ImageResource will - typically - still be based on the content of
the resource.
Examples
--------
>>> import imageio.v3 as iio
>>> with iio.imopen("/path/to/image.png", "r") as file:
>>> im = file.read()
>>> with iio.imopen("/path/to/output.jpg", "w") as file:
>>> file.write(im)
"""
if isinstance(uri, Request) and legacy_mode:
warnings.warn(
"`iio.core.Request` is a low-level object and using it"
" directly as input to `imopen` is discouraged. This will raise"
" an exception in ImageIO v3.",
DeprecationWarning,
stacklevel=2,
)
request = uri
uri = request.raw_uri
io_mode = request.mode.io_mode
request.format_hint = format_hint
else:
request = Request(uri, io_mode, format_hint=format_hint, extension=extension)
source = "<bytes>" if isinstance(uri, bytes) else uri
# fast-path based on plugin
# (except in legacy mode)
if plugin is not None:
if isinstance(plugin, str):
try:
config = known_plugins[plugin]
except KeyError:
request.finish()
raise ValueError(
f"`{plugin}` is not a registered plugin name."
) from None
def loader(request, **kwargs):
return config.plugin_class(request, **kwargs)
elif not legacy_mode:
def loader(request, **kwargs):
return plugin(request, **kwargs)
else:
request.finish()
raise ValueError("The `plugin` argument must be a string.")
try:
return loader(request, **kwargs)
except InitializationError as class_specific:
err_from = class_specific
err_type = RuntimeError if legacy_mode else IOError
err_msg = f"`{plugin}` can not handle the given uri."
except ImportError:
err_from = None
err_type = ImportError
err_msg = (
f"The `{config.name}` plugin is not installed. "
f"Use `pip install imageio[{config.install_name}]` to install it."
)
except Exception as generic_error:
err_from = generic_error
err_type = IOError
err_msg = f"An unknown error occured while initializing plugin `{plugin}`."
request.finish()
raise err_type(err_msg) from err_from
# fast-path based on format_hint
if request.format_hint is not None:
for candidate_format in known_extensions[format_hint]:
for plugin_name in candidate_format.priority:
config = known_plugins[plugin_name]
# v2 compatibility; delete in v3
if legacy_mode and not config.is_legacy:
continue
try:
candidate_plugin = config.plugin_class
except ImportError:
# not installed
continue
try:
plugin_instance = candidate_plugin(request, **kwargs)
except InitializationError:
# file extension doesn't match file type
continue
return plugin_instance
else:
resource = (
"<bytes>" if isinstance(request.raw_uri, bytes) else request.raw_uri
)
warnings.warn(f"`{resource}` can not be opened as a `{format_hint}` file.")
# fast-path based on file extension
if request.extension in known_extensions:
for candidate_format in known_extensions[request.extension]:
for plugin_name in candidate_format.priority:
config = known_plugins[plugin_name]
# v2 compatibility; delete in v3
if legacy_mode and not config.is_legacy:
continue
try:
candidate_plugin = config.plugin_class
except ImportError:
# not installed
continue
try:
plugin_instance = candidate_plugin(request, **kwargs)
except InitializationError:
# file extension doesn't match file type
continue
return plugin_instance
# error out for read-only special targets
# this is hacky; can we come up with a better solution for this?
if request.mode.io_mode == IOMode.write:
if isinstance(uri, str) and uri.startswith(SPECIAL_READ_URIS):
request.finish()
err_type = ValueError if legacy_mode else IOError
err_msg = f"`{source}` is read-only."
raise err_type(err_msg)
# error out for directories
# this is a bit hacky and should be cleaned once we decide
# how to gracefully handle DICOM
if request._uri_type == URI_FILENAME and Path(request.raw_uri).is_dir():
request.finish()
err_type = ValueError if legacy_mode else IOError
err_msg = (
"ImageIO does not generally support reading folders. "
"Limited support may be available via specific plugins. "
"Specify the plugin explicitly using the `plugin` kwarg, e.g. `plugin='DICOM'`"
)
raise err_type(err_msg)
# close the current request here and use fresh/new ones while trying each
# plugin This is slow (means potentially reopening a resource several
# times), but should only happen rarely because this is the fallback if all
# else fails.
request.finish()
# fallback option: try all plugins
for config in known_plugins.values():
# Note: for v2 compatibility
# this branch can be removed in ImageIO v3.0
if legacy_mode and not config.is_legacy:
continue
# each plugin gets its own request
request = Request(uri, io_mode, format_hint=format_hint)
try:
plugin_instance = config.plugin_class(request, **kwargs)
except InitializationError:
continue
except ImportError:
continue
else:
return plugin_instance
err_type = ValueError if legacy_mode else IOError
err_msg = f"Could not find a backend to open `{source}`` with iomode `{io_mode}`."
# check if a missing plugin could help
if request.extension in known_extensions:
missing_plugins = list()
formats = known_extensions[request.extension]
plugin_names = [
plugin for file_format in formats for plugin in file_format.priority
]
for name in plugin_names:
config = known_plugins[name]
try:
config.plugin_class
continue
except ImportError:
missing_plugins.append(config)
if len(missing_plugins) > 0:
install_candidates = "\n".join(
[
(
f" {config.name}: "
f"pip install imageio[{config.install_name}]"
)
for config in missing_plugins
]
)
err_msg += (
"\nBased on the extension, the following plugins might add capable backends:\n"
f"{install_candidates}"
)
request.finish()
raise err_type(err_msg)
E ValueError: Could not find a backend to open ggg.mp4`` with iomode w?`.
E Based on the extension, the following plugins might add capable backends:
E FFMPEG: pip install imageio[ffmpeg]
E pyav: pip install imageio[pyav]
../../../.pyenv/versions/3.10.1/lib/python3.10/site-packages/imageio/core/imopen.py:298: ValueError
======================== 1 failed, 4 warnings in 0.67s =========================
Process finished with exit code 1
我写了一个例子，使用imageio结合WDA的mjpeg服务器录屏，你可以参考一下（因为是通过图片合成的视频，理论上是跨平台的 https://github.com/alibaba/taobao-iphone-设备/blob/d6071b6236976dcfd469a5175b019a9aeef209f6/scripts/uitest_screenrecord.py
hello，我在使用时发现，生成的录屏视频是变速的，原本录屏时花了40s，但实际录屏文件只有15s，请问有什么办法可以解决吗？感谢！！
