🔧 iOS Obj-C++ Frame Processor Plugin Build Failed: 'FrameHostObject.h' not found

[MatrixTurtle269]

How were you trying to build the app?

I tried to build my Frame Processor Plugin (written in Objective-C++), but it failed with 'FrameHostObject.h' file not found. The issue seems to appear in VisionCamera/FrameProcessors/FrameProcessor.h:

// FrameProcessor.h

#pragma once

#import "Frame.h"
#import <AVFoundation/AVFoundation.h>
#import <Foundation/Foundation.h>

#ifdef __cplusplus
#import "FrameHostObject.h" >>> 'FrameHostObject.h' file not found
#import "WKTJsiWorklet.h"
#import <jsi/jsi.h>
#import <memory.h>
#endif

NS_ASSUME_NONNULL_BEGIN

@interface FrameProcessor : NSObject

- (instancetype)init NS_UNAVAILABLE;

#ifdef __cplusplus
- (instancetype _Nonnull)initWithWorklet:(std::shared_ptr<RNWorklet::JsiWorklet>)worklet
                                 context:(std::shared_ptr<RNWorklet::JsiWorkletContext>)context;

- (void)callWithFrameHostObject:(std::shared_ptr<FrameHostObject>)frameHostObject;
#endif

- (void)call:(Frame*)frame;

@end

NS_ASSUME_NONNULL_END

This is weird, since the Xcode editor seems to recognize FrameHostObject.h just fine.

Reopening #1682 since it was marked as stale.

Full build logs

Showing Recent Errors Only

...


In file included from /Users/user/Projects/project/ios/LGAFrameProcessorPlugin.mm:8:
In file included from /Users/user/Projects/project/ios/Pods/Headers/Public/VisionCamera/FrameProcessorPlugin.h:12:
In file included from /Users/user/Projects/project/ios/Pods/Headers/Public/VisionCamera/FrameProcessorPluginRegistry.h:13:
In file included from /Users/user/Projects/project/ios/Pods/Headers/Public/VisionCamera/VisionCameraProxyHolder.h:10:
/Users/user/Projects/project/ios/Pods/Headers/Public/VisionCamera/FrameProcessor.h:16:9: fatal error: 'FrameHostObject.h' file not found
#import "FrameHostObject.h"
        ^~~~~~~~~~~~~~~~~~~
7 warnings and 1 error generated.

/Users/user/Projects/project/ios/Pods/Headers/Public/VisionCamera/FrameProcessor.h:16:9: 'FrameHostObject.h' file not found



Activity Log Complete    7/9/24, 12:34 AM    8.3 seconds

Project dependencies

"dependencies": {
    "@expo/vector-icons": "^14.0.2",
    "@react-navigation/native": "^6.0.2",
    "@shopify/react-native-skia": "^1.3.7",
    "expo": "~51.0.17",
    "expo-font": "~12.0.7",
    "expo-linking": "~6.3.1",
    "expo-router": "~3.5.17",
    "expo-splash-screen": "~0.27.5",
    "expo-status-bar": "~1.12.1",
    "expo-system-ui": "~3.0.6",
    "expo-web-browser": "~13.0.3",
    "react": "18.2.0",
    "react-dom": "18.2.0",
    "react-native": "0.74.2",
    "react-native-fast-tflite": "^1.2.0",
    "react-native-paper": "^5.12.3",
    "react-native-reanimated": "~3.10.1",
    "react-native-safe-area-context": "4.10.1",
    "react-native-screens": "3.31.1",
    "react-native-vision-camera": "^4.3.2",
    "react-native-web": "~0.19.10",
    "react-native-worklets-core": "^1.3.3",
    "vision-camera-resize-plugin": "^3.1.0"
  },

VisionCamera Version

4.4.1

Target platforms

iOS

Operating system

MacOS

Can you build the VisionCamera Example app?

Yes, I can successfully build the Example app here

Additional information

• [ ] I am using Expo
• [X] I have enabled Frame Processors (react-native-worklets-core)
• [X] I have read the Troubleshooting Guide
• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar issues in this repository and found none.

[maintenance-hans[bot]]

Guten Tag, Hans here.

[!NOTE] New features, bugfixes, updates and other improvements are all handled mostly by @mrousavy in his free time. To support @mrousavy, please consider 💖 sponsoring him on GitHub 💖. Sponsored issues will be prioritized.

[mrousavy]

That's because the FrameHostObject is not exposed, it's a private API. Why would you need that?

[MatrixTurtle269]

@mrousavy Well, I can't build my frame processor plugin because of this. I have to write it in Objective-C++ (.mm) because of a dependency. Is there a way to make this work?

[mrousavy]

Ahh, now I get it. I thought you were including FrameHostObject.h yourself.

Okay I see the issue. I'll think about this a bit, but this is not easy to fix. I assume __cplusplus only for internal usage.

[mrousavy]

As a workaround, you can just build an Objective-C Frame Processor plugin, which then later calls into Objective-C++ inside it's callback body.

[MatrixTurtle269]

Ahh, now I get it. I thought you were including FrameHostObject.h yourself.

Yeah, sorry for the confusing description.

I'll just build the plugin in Swift and write necessary methods in Obj-C++ for now.

[mrousavy]

In the future everything will be Swift. It's the direction I want to move towards with VisionCamera.

With Swift 5.9/6 you can call a lot of C++ methods directly anyways, no need for Objective-C(++) here. Also, Swift method calls are faster than Objective-C method calls ("message sends").

[MatrixTurtle269]

The only reason I need to use Objective-C++ at all is because I need the OpenCV framework, which only seems to work with Objective-C++ as of now.

I'm glad you're planning to prioritize Swift in the future, it's been a nightmare trying to learn Objective-C syntax tbh

[mrousavy]

I think OpenCV can be used from Swift. Either directly, or with some simple C++ bridging code. Either way, this should be a tiny bit faster than Objective-C.

[bdtren]

In my case,

I added this to podfile target to make sure react-native-vision-camera is linkage static:

  static_frameworks = ['react-native-vision-camera']

  pre_install do |installer|
      if static_frameworks.include?(pod.name)
        def pod.build_type
          Pod::BuildType.static_framework
        end
      end
    end
  end

Then in the .h file, I added:

#import <VisionCamera/FrameProcessorPlugin.h>

And finally, I open XCode, choose target, go to Build Settings → Header Search Paths and add:

$(PODS_ROOT)/Headers/Private/VisionCamera

And the issue is fixed.