idb icon indicating copy to clipboard operation
idb copied to clipboard

Published 20 hours ago • verified publisher icon facebook

Can't install idb-companion on macOS Catalina and BigSur

Open designervoid opened this issue 3 years ago • 24 comments

Description

tried to install idb-companion on macOS Catalina and macOS Big Sur and having compilation issues. Here is the log (installation via homebrew or via source code):

** BUILD FAILED **


The following build commands failed:
    CompileC /tmp/idb-companion-20210520-97887-1d577si/idb-1.1.2/build/Build/Intermediates.noindex/idb_companion.build/Debug/idb_companion.build/Objects-normal/x86_64/FBIDBServiceHandler.o /tmp/idb-companion-20210520-97887-1d577si/idb-1.1.2/idb_companion/Server/FBIDBServiceHandler.mm normal x86_64 objective-c++ com.apple.compilers.llvm.clang.1_0.compiler
(1 failure)

Reproduction

Run these commands on Catalina 10.15.7 or Big Sur 11.2. Homebrew:

$ brew tap facebook/fb
$ brew install idb-companion

Source code:

$ git clone https://github.com/facebook/idb.git
$ cd idb
$ ./idb_build.sh idb_companion build /usr/local/bin/idb_companion

Solution

https://github.com/facebook/idb/issues/671#issuecomment-1000497933

Additional Information

...

designervoid avatar May 20 '21 15:05 designervoid

tried updating to the latest version (macOS BigSur 11.3.1). also removed the package idb-companion:

designervoid@user idb % brew uninstall idb-companion
Uninstalling /usr/local/Cellar/idb-companion/1.1.2... (222 files, 22.3MB)

and installed again:

designervoid@user idb % brew tap facebook/fb
designervoid@user idb % brew install idb-companion
==> Installing idb-companion from facebook/fb
==> Downloading https://github.com/facebook/idb/releases/download/v1.1.2/idb-companion-1.1.2.big_sur.bottle.tar.gz
Already downloaded: /Users/designervoid/Library/Caches/Homebrew/downloads/a145467f25843b711ae458845a58046650f93d6df57cb1a2ecca7777fc9446c3--idb-companion-1.1.2.big_sur.bottle.tar.gz
==> Pouring idb-companion-1.1.2.big_sur.bottle.tar.gz
==> codesign --force --sign - --timestamp=none /usr/local/Cellar/idb-companion/1.1.2/Frameworks/FBDeviceControl.framework/Versions/A/Resources/libShimulator.d
==> codesign --force --sign - --timestamp=none /usr/local/Cellar/idb-companion/1.1.2/Frameworks/FBSimulatorControl.framework/Versions/A/Resources/libShimulato
==> codesign --force --sign - --timestamp=none /usr/local/Cellar/idb-companion/1.1.2/Frameworks/XCTestBootstrap.framework/Versions/A/Resources/libShimulator.d
🍺  /usr/local/Cellar/idb-companion/1.1.2: 222 files, 22.3MB
designervoid@user idb % ls /usr/local/bin

result of ls command: ls /usr/local/bin image trying to install fb-idb pip package:

designervoid@user idb % python3 --version
Python 3.9.5
designervoid@user idb % sudo -H pip3 install fb-idb 
Collecting fb-idb
  Downloading fb_idb-1.1.2-py3-none-any.whl (143 kB)
     |████████████████████████████████| 143 kB 1.2 MB/s 
Requirement already satisfied: grpclib>=0.4.0 in /usr/local/lib/python3.9/site-packages (from fb-idb) (0.4.1)
Requirement already satisfied: aiofiles in /usr/local/lib/python3.9/site-packages (from fb-idb) (0.7.0)
Requirement already satisfied: protobuf in /usr/local/lib/python3.9/site-packages (from fb-idb) (3.17.0)
Requirement already satisfied: treelib in /usr/local/lib/python3.9/site-packages (from fb-idb) (1.6.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.9/site-packages (from grpclib>=0.4.0->fb-idb) (5.1.0)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.9/site-packages (from grpclib>=0.4.0->fb-idb) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.9/site-packages (from h2<5,>=3.1.0->grpclib>=0.4.0->fb-idb) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.9/site-packages (from h2<5,>=3.1.0->grpclib>=0.4.0->fb-idb) (4.0.0)
Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.9/site-packages (from protobuf->fb-idb) (1.16.0)
Requirement already satisfied: future in /usr/local/lib/python3.9/site-packages (from treelib->fb-idb) (0.18.2)
Installing collected packages: fb-idb
Successfully installed fb-idb-1.1.2
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv

Make sure the package is installed:

designervoid@user idb % which idb
/usr/local/bin/idb

Now we are trying to run the command of idb package:

designervoid@user idb % idb list-targets 
2021-05-20 21:43:28,148 [ERROR] - root - Exception thrown in main
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion.py", line 71, in parse_json_line
    return json.loads(decoded_line)
  File "/usr/local/Cellar/[email protected]/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/[email protected]/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/[email protected]/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion_spawner.py", line 153, in spawn_notifier
    await self._read_notifier_output(stream=none_throws(process.stdout))
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion_spawner.py", line 166, in _read_notifier_output
    update = parse_json_line(line)
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion.py", line 73, in parse_json_line
    raise IdbJsonException(f"Failed to parse json from: {decoded_line}")
idb.common.companion.IdbJsonException: Failed to parse json from: 

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/idb/cli/main.py", line 287, in gen_main
    await root_command.run(args)
  File "/usr/local/lib/python3.9/site-packages/idb/common/command.py", line 90, in run
    return await self.resolve_command_from_args(args).run(args)
  File "/usr/local/lib/python3.9/site-packages/idb/cli/__init__.py", line 106, in run
    await self._run_impl(args)
  File "/usr/local/lib/python3.9/site-packages/idb/cli/__init__.py", line 149, in _run_impl
    await self.run_with_manager(
  File "/usr/local/lib/python3.9/site-packages/idb/cli/commands/target.py", line 171, in run_with_manager
    targets = await manager.list_targets()
  File "/usr/local/lib/python3.9/site-packages/idb/common/logging.py", line 113, in _async_wrapper
    raise ex
  File "/usr/local/lib/python3.9/site-packages/idb/common/logging.py", line 88, in _async_wrapper
    value = await function(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/idb/grpc/management.py", line 135, in list_targets
    await self._spawn_notifier()
  File "/usr/local/lib/python3.9/site-packages/idb/grpc/management.py", line 57, in _spawn_notifier
    await companion_spawner.spawn_notifier()
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion_spawner.py", line 156, in spawn_notifier
    raise CompanionSpawnerException(
idb.common.companion_spawner.CompanionSpawnerException: Failed to spawn the idb notifier. Stderr: dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

designervoid avatar May 20 '21 18:05 designervoid

also information about idb-companion:

designervoid@user idb % ls -l `which idb_companion`
lrwxr-xr-x  1 designervoid  admin  47 20 may 21:35 /usr/local/bin/idb_companion -> ../Cellar/idb-companion/1.1.2/bin/idb_companion

designervoid avatar May 20 '21 18:05 designervoid

I am having the exact same issue. Running Big Sur 11.3.1. I also attempted to build idb from source and ran into build issues there.

L-Yeiser avatar May 20 '21 19:05 L-Yeiser

Also tried build from source in XCode: image

designervoid avatar May 20 '21 19:05 designervoid

@designervoid that is the error I was getting from xCode.

L-Yeiser avatar May 20 '21 19:05 L-Yeiser 

/Users/designervoid/Desktop/tests/idb/idb_companion/Server/FBIDBServiceHandler.mm:100:14: error: no matching member function for call to 'set_data'
    payload->set_data(buffer, size);
    ~~~~~~~~~^~~~~~~~
In file included from /Users/designervoid/Desktop/tests/idb/idb_companion/Server/FBIDBServiceHandler.mm:10:
In file included from /Users/designervoid/Library/Developer/Xcode/DerivedData/idb_companion-azrywzqdgdfzqhfuhgnzcgijymvs/Build/Products/Debug/idbGRPC/idb.grpc.pb.h:14:
/Users/designervoid/Library/Developer/Xcode/DerivedData/idb_companion-azrywzqdgdfzqhfuhgnzcgijymvs/Build/Products/Debug/idbGRPC/idb.pb.h:21438:22: note: candidate template ignored: substitution failure: variably modified type 'unsigned long (&)[BufferOutputSize]' cannot be used as a template argument
inline void Payload::set_data(ArgT0&& arg0, ArgT... args) {

log says that file /Users/designervoid/Desktop/tests/idb/idb_companion/Server/FBIDBServiceHandler.mm: has no payload with set_data function

designervoid avatar May 20 '21 22:05 designervoid

Same issue here. I will try to lower down grpc version and build from source No luck with [email protected]

tomasmozeris avatar May 21 '21 06:05 tomasmozeris

tried updating to the latest version (macOS BigSur 11.3.1). also removed the package idb-companion:

maximchernyshov@MacBook-Pro-Maxim-2 idb % brew uninstall idb-companion
Uninstalling /usr/local/Cellar/idb-companion/1.1.2... (222 files, 22.3MB)

and installed again:

maximchernyshov@MacBook-Pro-Maxim-2 idb % brew tap facebook/fb
maximchernyshov@MacBook-Pro-Maxim-2 idb % brew install idb-companion
==> Installing idb-companion from facebook/fb
==> Downloading https://github.com/facebook/idb/releases/download/v1.1.2/idb-companion-1.1.2.big_sur.bottle.tar.gz
Already downloaded: /Users/maximchernyshov/Library/Caches/Homebrew/downloads/a145467f25843b711ae458845a58046650f93d6df57cb1a2ecca7777fc9446c3--idb-companion-1.1.2.big_sur.bottle.tar.gz
==> Pouring idb-companion-1.1.2.big_sur.bottle.tar.gz
==> codesign --force --sign - --timestamp=none /usr/local/Cellar/idb-companion/1.1.2/Frameworks/FBDeviceControl.framework/Versions/A/Resources/libShimulator.d
==> codesign --force --sign - --timestamp=none /usr/local/Cellar/idb-companion/1.1.2/Frameworks/FBSimulatorControl.framework/Versions/A/Resources/libShimulato
==> codesign --force --sign - --timestamp=none /usr/local/Cellar/idb-companion/1.1.2/Frameworks/XCTestBootstrap.framework/Versions/A/Resources/libShimulator.d
🍺  /usr/local/Cellar/idb-companion/1.1.2: 222 files, 22.3MB
maximchernyshov@MacBook-Pro-Maxim-2 idb % ls /usr/local/bin

result of ls command: image trying to install fb-idb pip package:

maximchernyshov@MacBook-Pro-Maxim-2 idb % python3 --version
Python 3.9.5
maximchernyshov@MacBook-Pro-Maxim-2 idb % sudo -H pip3 install fb-idb 
Collecting fb-idb
  Downloading fb_idb-1.1.2-py3-none-any.whl (143 kB)
     |████████████████████████████████| 143 kB 1.2 MB/s 
Requirement already satisfied: grpclib>=0.4.0 in /usr/local/lib/python3.9/site-packages (from fb-idb) (0.4.1)
Requirement already satisfied: aiofiles in /usr/local/lib/python3.9/site-packages (from fb-idb) (0.7.0)
Requirement already satisfied: protobuf in /usr/local/lib/python3.9/site-packages (from fb-idb) (3.17.0)
Requirement already satisfied: treelib in /usr/local/lib/python3.9/site-packages (from fb-idb) (1.6.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.9/site-packages (from grpclib>=0.4.0->fb-idb) (5.1.0)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.9/site-packages (from grpclib>=0.4.0->fb-idb) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.9/site-packages (from h2<5,>=3.1.0->grpclib>=0.4.0->fb-idb) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.9/site-packages (from h2<5,>=3.1.0->grpclib>=0.4.0->fb-idb) (4.0.0)
Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.9/site-packages (from protobuf->fb-idb) (1.16.0)
Requirement already satisfied: future in /usr/local/lib/python3.9/site-packages (from treelib->fb-idb) (0.18.2)
Installing collected packages: fb-idb
Successfully installed fb-idb-1.1.2
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv

Make sure the package is installed:

maximchernyshov@MacBook-Pro-Maxim-2 idb % which idb
/usr/local/bin/idb

Now we are trying to run the command of idb package:

maximchernyshov@MacBook-Pro-Maxim-2 idb % idb list-targets 
2021-05-20 21:43:28,148 [ERROR] - root - Exception thrown in main
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion.py", line 71, in parse_json_line
    return json.loads(decoded_line)
  File "/usr/local/Cellar/[email protected]/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/[email protected]/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/[email protected]/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion_spawner.py", line 153, in spawn_notifier
    await self._read_notifier_output(stream=none_throws(process.stdout))
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion_spawner.py", line 166, in _read_notifier_output
    update = parse_json_line(line)
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion.py", line 73, in parse_json_line
    raise IdbJsonException(f"Failed to parse json from: {decoded_line}")
idb.common.companion.IdbJsonException: Failed to parse json from: 

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/idb/cli/main.py", line 287, in gen_main
    await root_command.run(args)
  File "/usr/local/lib/python3.9/site-packages/idb/common/command.py", line 90, in run
    return await self.resolve_command_from_args(args).run(args)
  File "/usr/local/lib/python3.9/site-packages/idb/cli/__init__.py", line 106, in run
    await self._run_impl(args)
  File "/usr/local/lib/python3.9/site-packages/idb/cli/__init__.py", line 149, in _run_impl
    await self.run_with_manager(
  File "/usr/local/lib/python3.9/site-packages/idb/cli/commands/target.py", line 171, in run_with_manager
    targets = await manager.list_targets()
  File "/usr/local/lib/python3.9/site-packages/idb/common/logging.py", line 113, in _async_wrapper
    raise ex
  File "/usr/local/lib/python3.9/site-packages/idb/common/logging.py", line 88, in _async_wrapper
    value = await function(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/idb/grpc/management.py", line 135, in list_targets
    await self._spawn_notifier()
  File "/usr/local/lib/python3.9/site-packages/idb/grpc/management.py", line 57, in _spawn_notifier
    await companion_spawner.spawn_notifier()
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion_spawner.py", line 156, in spawn_notifier
    raise CompanionSpawnerException(
idb.common.companion_spawner.CompanionSpawnerException: Failed to spawn the idb notifier. Stderr: dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.dylib

  Referenced from: /usr/local/bin/idb_companion

  Reason: image not found

@designervoid Had the same issue as you. After many hours I tried doing something pretty dirty that unlocked me a little: I went to /usr/local/opt/grpc/lib/, figured I had a libgrpc++.1.38.dylib file but no libgrpc++.1.dylib, so I duplicated the former and renamed the duplicate libgrpc++.1.dylib, tried again which made more similar errors pop, which I solved using the same strategy.

After that I could run idb list-targets successfully. While this is probably the wrong way to fix the issue, it makes me feel like idb and grpc are desynchronised.

Further to that I tried idb connect --udid <my_target_udid> which failed with the errors Failed to spawn companion, couldn't read port and Failed to start GRPC server.

So that didn't bring me very far because idb_companion is anyway wrongly set up.

Finally I tried building idb_companion from source and got the same issues as mentioned in this thread, build failed with set_data error. I also tried checking out in v1.1.0 but got the same error. I'm guessing this has more to do with the version of XCode / big sur than the code itself.

Hope this helps someone to unlock the situation, I'm looking forward to leveraging Flipper in my daily work!

maxime-memtell avatar May 23 '21 13:05 maxime-memtell

Same issue here. I will try to lower down grpc version and build from source No luck with [email protected]

Hello, how to downgrade the version of grpc?

`dyld: Library not loaded: /usr/local/opt/grpc/lib/libgrpc.15.dylib

Referenced from: /usr/local/bin/idb_companion

Reason: image not found`

zLinz avatar May 25 '21 03:05 zLinz 

/Users/maximchernyshov/Desktop/tests/idb/idb_companion/Server/FBIDBServiceHandler.mm:100:14: error: no matching member function for call to 'set_data'
    payload->set_data(buffer, size);
    ~~~~~~~~~^~~~~~~~
In file included from /Users/maximchernyshov/Desktop/tests/idb/idb_companion/Server/FBIDBServiceHandler.mm:10:
In file included from /Users/maximchernyshov/Library/Developer/Xcode/DerivedData/idb_companion-azrywzqdgdfzqhfuhgnzcgijymvs/Build/Products/Debug/idbGRPC/idb.grpc.pb.h:14:
/Users/maximchernyshov/Library/Developer/Xcode/DerivedData/idb_companion-azrywzqdgdfzqhfuhgnzcgijymvs/Build/Products/Debug/idbGRPC/idb.pb.h:21438:22: note: candidate template ignored: substitution failure: variably modified type 'unsigned long (&)[BufferOutputSize]' cannot be used as a template argument
inline void Payload::set_data(ArgT0&& arg0, ArgT... args) {

log says that file /Users/maximchernyshov/Desktop/tests/idb/idb_companion/Server/FBIDBServiceHandler.mm: has no payload with set_data function

Hey guys!

in the set_data error; i change to this code, use the NSData to replace buffer!!!

FBIDBServiceHandler.mm

` uintptr_t buffer[BufferOutputSize]; NSInteger size = [inputStream read:(uint8_t *)buffer maxLength:BufferOutputSize]; // use the NSData to replace buffer NSData *data = [NSData dataWithBytes:buffer length:BufferOutputSize]; if (size == 0) { break; } if (size < 0) { return Status::OK; } T response; idb::Payload *payload = response.mutable_payload(); payload->set_data(data.bytes, data.length); stream->Write(response);

`

and than in command line IDB_COMPANION=localhost:10882 idb launch com.apple.reminders

IT WORKS!!!

I'm not sure this is the best workaround ,but i can run idb now!!!

zLinz avatar May 25 '21 05:05 zLinz

@zLinz thanks for that, it did help me pass the build step, I've been fighthing on this for 2 days now (reinstalled everything from brew / python / macos helped get passed a bunch of different issues with grcp etc, but then I ended up with the JSonDecode error when trying to run from brew / pip package, went on to try to build from source, found this issue ... this is a nightmare devXP, I assume I'm using it too soon and it's not ready yet).

So although this build without issue, it fails when trying to run (with a simulator device target):

2021-05-25 10:22:17.675015+0200 idb_companion[17315:144986] tid:103 - Successfully registered new client
2021-05-25 10:22:17.675343+0200 idb_companion[17315:144986] tid:103 - Now tracking 1 clients
2021-05-25 10:22:17.748838+0200 idb_companion[17315:144986] [simulator_set] connectToBridge called with: []
2021-05-25 10:22:17.762757+0200 idb_companion[17315:145673] [simulator_set] connectToBridge failed with: Some idb functionality is not yet available with Xcode 12.5, downgrade your Xcode version and try again.
2021-05-25 10:22:17.775821+0200 idb_companion[17315:144986] [] Some idb functionality is not yet available with Xcode 12.5, downgrade your Xcode version and try again.
Program ended with exit code: 1

Did you downgrade XCode or did not encounter this issue ? As shown in the error logs, I'm on XCode 12.5 / bigSur 11.3.1.

matgardon avatar May 25 '21 08:05 matgardon

@zLinz That works indeed!

I'm going to assume this is a better fix than me commenting the faulty line (which works, believe me, I tried 😄 ...)

@matgardon FYI I did not encounter this issue so far and I'm using the same config. And I had to, since I built on a target using ios 14.5.1 for which you need xcode 12.5...

Just to recap, the steps I took that allowed me to work with flipper (with errors about csrf) and use idb and idb_companion to connect to a real target:

  • I made sure brew was up to date and that I was using the latest version of Python and pip (3.9 and 21.1.1 respectively)
  • I installed the latest version of grpc
  • I installed the latest version of idb with the --user flag
  • I downloaded the idb project and: ** applied the fix mentioned by zLinz ** opened the project in Xcode ** changed the Embed mode of the .framework files to "Embed Without Signing" ** added to the Runpath Search Paths (in Targets > idb_companion > Build Settings > Linking) the following paths: @executable_path/../Frameworks @executable_path/Frameworks @loader_path/../Frameworks ** Changed the "Build Active Architecture Only" Flag to "Yes" (Projects>idb_companion>Build settings>Architecture)
  • archived the idb_companion target, then "Distribute" as "Built Products"
  • copy and pasted the resulting idb_companion file in my /usr/local/bin folder and the Frameworks in /usr/local/Frameworks
  • made sure that my Path contains my local Python version I use

Honestly, I have no idea which of these points are the ones that made the difference. Just sharing in the hope that it helps some people to unlock the situation because I did also spend 2 days straight trying absolutely everything...

EDIT: CSRF issue solved. So after checking https://github.com/facebook/flipper/issues/2113 I realised that I had done one more step that I did not describe here and that turned out to be a mistake: I added in the scheme's build's arguments passed on launch specific udid's (potential solution I saw on SO I believe). Once I reverted this change, if I idb connect <my currently connected device's udid> before launching flipper, I'm fine

maxime-memtell avatar May 25 '21 09:05 maxime-memtell

@zLinz after doing what you suggested. I'm trying to archive the product. and I get the following errors.

  1. idb_companion.xcodeproj The linked and embedded library 'libgrpc.16.0.0.dylib' is missing one or more architectures required by this target: arm64.
  2. idb_companion.xcodeproj The linked library 'libprotobuf.a' is missing one or more architectures required by this target: arm64.
  3. idb_companion.xcodeproj The linked and embedded library 'libgrpc++.1.38.0.dylib' is missing one or more architectures required by this target: arm64.

sachinhr10 avatar May 25 '21 11:05 sachinhr10

@sachinhr10 On my computer the files you're looking for are currently in the /usr/local/Cellar/grpc/1.38.0/lib folder. Did you try running brew install grpc? Also I have a link to them in /usr/local/opt/grpc/lib/ which I assume brew made by itself

maxime-memtell avatar May 25 '21 11:05 maxime-memtell

Alright my bad, you did otherwise you wouldn't get this error.

I double checked and I actually used one of the solutions provided here https://github.com/facebook/react-native/issues/29984 and set the "Build Active Architecture Only" flag to "Yes". I'll update the steps described above.

As I said I really stumbled in the dark for two days before something happened so I might both have missed some info and/or provided superfluous steps :/ I'm really hoping there's a PR that fixes all of this properly

maxime-memtell avatar May 25 '21 11:05 maxime-memtell

@zLinz thanks for that, it did help me pass the build step, I've been fighthing on this for 2 days now (reinstalled everything from brew / python / macos helped get passed a bunch of different issues with grcp etc, but then I ended up with the JSonDecode error when trying to run from brew / pip package, went on to try to build from source, found this issue ... this is a nightmare devXP, I assume I'm using it too soon and it's not ready yet).

So although this build without issue, it fails when trying to run (with a simulator device target):

2021-05-25 10:22:17.675015+0200 idb_companion[17315:144986] tid:103 - Successfully registered new client
2021-05-25 10:22:17.675343+0200 idb_companion[17315:144986] tid:103 - Now tracking 1 clients
2021-05-25 10:22:17.748838+0200 idb_companion[17315:144986] [simulator_set] connectToBridge called with: []
2021-05-25 10:22:17.762757+0200 idb_companion[17315:145673] [simulator_set] connectToBridge failed with: Some idb functionality is not yet available with Xcode 12.5, downgrade your Xcode version and try again.
2021-05-25 10:22:17.775821+0200 idb_companion[17315:144986] [] Some idb functionality is not yet available with Xcode 12.5, downgrade your Xcode version and try again.
Program ended with exit code: 1

Did you downgrade XCode or did not encounter this issue ? As shown in the error logs, I'm on XCode 12.5 / bigSur 11.3.1.

I did not downgrade anything. I'm on XCode 12.5 / bigSur 11.3.1, too.
and i upgrade to bigsur 11.4 last night. it also works.

zLinz avatar May 26 '21 02:05 zLinz

@zLinz thanks for that, it did help me pass the build step, I've been fighthing on this for 2 days now (reinstalled everything from brew / python / macos helped get passed a bunch of different issues with grcp etc, but then I ended up with the JSonDecode error when trying to run from brew / pip package, went on to try to build from source, found this issue ... this is a nightmare devXP, I assume I'm using it too soon and it's not ready yet).

So although this build without issue, it fails when trying to run (with a simulator device target):

2021-05-25 10:22:17.675015+0200 idb_companion[17315:144986] tid:103 - Successfully registered new client
2021-05-25 10:22:17.675343+0200 idb_companion[17315:144986] tid:103 - Now tracking 1 clients
2021-05-25 10:22:17.748838+0200 idb_companion[17315:144986] [simulator_set] connectToBridge called with: []
2021-05-25 10:22:17.762757+0200 idb_companion[17315:145673] [simulator_set] connectToBridge failed with: Some idb functionality is not yet available with Xcode 12.5, downgrade your Xcode version and try again.
2021-05-25 10:22:17.775821+0200 idb_companion[17315:144986] [] Some idb functionality is not yet available with Xcode 12.5, downgrade your Xcode version and try again.
Program ended with exit code: 1

Did you downgrade XCode or did not encounter this issue ? As shown in the error logs, I'm on XCode 12.5 / bigSur 11.3.1.

You should run idb_companion on my mac. what dose this mean? "it fails when trying to run (with a simulator device target)"

zLinz avatar May 26 '21 03:05 zLinz

@zLinz after doing what you suggested. I'm trying to archive the product. and I get the following errors.

  1. idb_companion.xcodeproj The linked and embedded library 'libgrpc.16.0.0.dylib' is missing one or more architectures required by this target: arm64.
  2. idb_companion.xcodeproj The linked library 'libprotobuf.a' is missing one or more architectures required by this target: arm64.
  3. idb_companion.xcodeproj The linked and embedded library 'libgrpc++.1.38.0.dylib' is missing one or more architectures required by this target: arm64.

I can find the lib in path:/usr/local/opt/grpc/lib/ I did not try to archive the product. Can you run without errors?

zLinz avatar May 26 '21 03:05 zLinz

https://github.com/facebook/idb/pull/672 This fix may help.

djs66256 avatar May 26 '21 07:05 djs66256

@zLinz after doing what you suggested. I'm trying to archive the product. and I get the following errors.

  1. idb_companion.xcodeproj The linked and embedded library 'libgrpc.16.0.0.dylib' is missing one or more architectures required by this target: arm64.
  2. idb_companion.xcodeproj The linked library 'libprotobuf.a' is missing one or more architectures required by this target: arm64.
  3. idb_companion.xcodeproj The linked and embedded library 'libgrpc++.1.38.0.dylib' is missing one or more architectures required by this target: arm64.

I can find the lib in path:/usr/local/opt/grpc/lib/ I did not try to archive the product. Can you run without errors?

I did some fixes and now I'm able to run without errors. But archiving doesn't work for me. I want idb-companion to work in /usr/local/bin. But even though I'm doing embed without signing to FBSimulatorControl, it says can't find the image. @maxime-memtell any idea on this ?

sachinhr10 avatar May 26 '21 08:05 sachinhr10

@sachinhr10 did you copy & paste the frameworks in /usr/local/Frameworks ? That's where they should be I believe (at least where they are on my computer right now)

maxime-memtell avatar May 26 '21 08:05 maxime-memtell

@sachinhr10 did you copy & paste the frameworks in /usr/local/Frameworks ? That's where they should be I believe (at least where they are on my computer right now)

I did that and it's working. thank you :))) Seriously this problem needs to be fixed with a PR. Such time consuming job.

sachinhr10 avatar May 26 '21 09:05 sachinhr10

properly working on macOS 12.1🥳 just install this with pip3 or homebrew

designervoid avatar Dec 23 '21 19:12 designervoid

I am not able to run the maestro test on MacOS: ventura 13.1; Got below errors:

java.io.IOException: Cannot run program "idb_companion": error=2, No such file or directory at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128) at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071) at maestro.cli.device.DeviceService.startIdbCompanion(DeviceService.kt:96) at maestro.cli.device.DeviceService.prepareDevice(DeviceService.kt:73) at maestro.cli.device.PickDeviceInteractor.pickDevice(PickDeviceInteractor.kt:26) at maestro.cli.session.MaestroSessionManager.selectDevice(MaestroSessionManager.kt:87) at maestro.cli.session.MaestroSessionManager.newSession(MaestroSessionManager.kt:48) at maestro.cli.command.TestCommand.call(TestCommand.kt:97) at maestro.cli.command.TestCommand.call(TestCommand.kt:39) at picocli.CommandLine.executeUserObject(CommandLine.java:1933) at picocli.CommandLine.access$1200(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332) at picocli.CommandLine$RunLast.handle(CommandLine.java:2326) at picocli.CommandLine$RunLast.handle(CommandLine.java:2291) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159) at maestro.cli.DisableAnsiMixin$Companion.executionStrategy(DisableAnsiMixin.kt:22) at picocli.CommandLine.execute(CommandLine.java:2058) at maestro.cli.AppKt.main(App.kt:119) Caused by: java.io.IOException: error=2, No such file or directory at java.base/java.lang.ProcessImpl.forkAndExec(Native Method) at java.base/java.lang.ProcessImpl.(ProcessImpl.java:340) at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:271) at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)

mkaurea avatar Jan 13 '23 13:01 mkaurea