fix: Rename getFrameProcessorPlugin to initFrameProcessorPlugin (#2038)
				
					
				
			* fix: Rename `getFrameProcessorPlugin` to `initFrameProcessorPlugin` * fix: Make nullable, add comments * Format * Update FrameProcessorPlugin.java * Update FrameProcessorPlugin.h * fix: Fix dead links * Call super constructor * Update ExampleFrameProcessorPlugin.java * fix: Init calls
This commit is contained in:
		| @@ -52,7 +52,7 @@ Frame Processors are JS functions that will be _workletized_ using [react-native | |||||||
|  |  | ||||||
| Frame Processor Plugins are native functions (written in Objective-C, Swift, C++, Java or Kotlin) that are injected into the VisionCamera JS-Runtime. They can be synchronously called from your JS Frame Processors (using JSI) without ever going over the bridge. Because VisionCamera provides an easy-to-use plugin API, you can easily create a Frame Processor Plugin yourself. Some examples include [Barcode Scanning](https://developers.google.com/ml-kit/vision/barcode-scanning), [Face Detection](https://developers.google.com/ml-kit/vision/face-detection), [Image Labeling](https://developers.google.com/ml-kit/vision/image-labeling), [Text Recognition](https://developers.google.com/ml-kit/vision/text-recognition) and more. | Frame Processor Plugins are native functions (written in Objective-C, Swift, C++, Java or Kotlin) that are injected into the VisionCamera JS-Runtime. They can be synchronously called from your JS Frame Processors (using JSI) without ever going over the bridge. Because VisionCamera provides an easy-to-use plugin API, you can easily create a Frame Processor Plugin yourself. Some examples include [Barcode Scanning](https://developers.google.com/ml-kit/vision/barcode-scanning), [Face Detection](https://developers.google.com/ml-kit/vision/face-detection), [Image Labeling](https://developers.google.com/ml-kit/vision/image-labeling), [Text Recognition](https://developers.google.com/ml-kit/vision/text-recognition) and more. | ||||||
|  |  | ||||||
| > Learn how to [create Frame Processor Plugins](frame-processors-plugins-overview), or check out the [example Frame Processor Plugin for iOS](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/ios/Frame%20Processor%20Plugins/Example%20Plugin%20(Swift)/ExamplePluginSwift.swift) or [Android](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/android/app/src/main/java/com/mrousavy/camera/example/ExampleFrameProcessorPlugin.java). | > Learn how to [create Frame Processor Plugins](frame-processors-plugins-overview), or check out the [example Frame Processor Plugin for iOS](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/ios/Frame%20Processor%20Plugins/Example%20Swift%20Plugin/ExampleSwiftFrameProcessor.swift) or [Android](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/android/app/src/main/java/com/mrousavy/camera/example/ExampleKotlinFrameProcessorPlugin.kt). | ||||||
|  |  | ||||||
| ### The `Frame` object | ### The `Frame` object | ||||||
|  |  | ||||||
|   | |||||||
| @@ -11,7 +11,7 @@ To make the Frame Processor Plugin available to the Frame Processor Worklet Runt | |||||||
| ```ts | ```ts | ||||||
| import { VisionCameraProxy, Frame } from 'react-native-vision-camera' | import { VisionCameraProxy, Frame } from 'react-native-vision-camera' | ||||||
|  |  | ||||||
| const plugin = VisionCameraProxy.getFrameProcessorPlugin('scanFaces') | const plugin = VisionCameraProxy.initFrameProcessorPlugin('scanFaces') | ||||||
|  |  | ||||||
| /** | /** | ||||||
|  * Scans faces. |  * Scans faces. | ||||||
|   | |||||||
| @@ -108,7 +108,7 @@ public class FaceDetectorFrameProcessorPluginPackage implements ReactPackage { | |||||||
| ``` | ``` | ||||||
|  |  | ||||||
| :::note | :::note | ||||||
| The Frame Processor Plugin will be exposed to JS through the `VisionCameraProxy` object. In this case, it would be `VisionCameraProxy.getFrameProcessorPlugin("detectFaces")`. | The Frame Processor Plugin will be exposed to JS through the `VisionCameraProxy` object. In this case, it would be `VisionCameraProxy.initFrameProcessorPlugin("detectFaces")`. | ||||||
| ::: | ::: | ||||||
|  |  | ||||||
| 6. Register the package in MainApplication.java | 6. Register the package in MainApplication.java | ||||||
| @@ -146,7 +146,7 @@ class FaceDetectorFrameProcessorPlugin(options: Map<String, Any>?): FrameProcess | |||||||
| } | } | ||||||
| ``` | ``` | ||||||
|  |  | ||||||
| 4. **Implement your Frame Processing.** See the [Example Plugin (Java)](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/android/app/src/main/java/com/mrousavy/camera/example/ExampleFrameProcessorPlugin.java) for reference. | 4. **Implement your Frame Processing.** See the [Example Plugin (Kotlin)](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/android/app/src/main/java/com/mrousavy/camera/example/ExampleKotlinFrameProcessorPlugin.kt) for reference. | ||||||
| 5. Create a new Kotlin file which registers the Frame Processor Plugin in a React Package, for the Face Detector plugin this file will be called `FaceDetectorFrameProcessorPluginPackage.kt`: | 5. Create a new Kotlin file which registers the Frame Processor Plugin in a React Package, for the Face Detector plugin this file will be called `FaceDetectorFrameProcessorPluginPackage.kt`: | ||||||
|  |  | ||||||
| ```kotlin | ```kotlin | ||||||
| @@ -176,7 +176,7 @@ class FaceDetectorFrameProcessorPluginPackage : ReactPackage { | |||||||
| ``` | ``` | ||||||
|  |  | ||||||
| :::note | :::note | ||||||
| The Frame Processor Plugin will be exposed to JS through the `VisionCameraProxy` object. In this case, it would be `VisionCameraProxy.getFrameProcessorPlugin("detectFaces")`. | The Frame Processor Plugin will be exposed to JS through the `VisionCameraProxy` object. In this case, it would be `VisionCameraProxy.initFrameProcessorPlugin("detectFaces")`. | ||||||
| ::: | ::: | ||||||
|  |  | ||||||
| 6. Register the package in MainApplication.java | 6. Register the package in MainApplication.java | ||||||
|   | |||||||
| @@ -51,7 +51,7 @@ For reference see the [CLI's docs](https://github.com/mateusz1913/vision-camera- | |||||||
|  |  | ||||||
| @implementation FaceDetectorFrameProcessorPlugin | @implementation FaceDetectorFrameProcessorPlugin | ||||||
|  |  | ||||||
| - (instancetype) initWithOptions:(NSDictionary*)options; { | - (instancetype) initWithOptions:(NSDictionary* _Nullable)options; { | ||||||
|   self = [super initWithOptions:options]; |   self = [super initWithOptions:options]; | ||||||
|   return self; |   return self; | ||||||
| } | } | ||||||
| @@ -71,10 +71,10 @@ VISION_EXPORT_FRAME_PROCESSOR(FaceDetectorFrameProcessorPlugin, detectFaces) | |||||||
| ``` | ``` | ||||||
|  |  | ||||||
| :::note | :::note | ||||||
| The Frame Processor Plugin will be exposed to JS through the `VisionCameraProxy` object. In this case, it would be `VisionCameraProxy.getFrameProcessorPlugin("detectFaces")`. | The Frame Processor Plugin will be exposed to JS through the `VisionCameraProxy` object. In this case, it would be `VisionCameraProxy.initFrameProcessorPlugin("detectFaces")`. | ||||||
| ::: | ::: | ||||||
|  |  | ||||||
| 4. **Implement your Frame Processing.** See the [Example Plugin (Objective-C)](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/ios/Frame%20Processor%20Plugins/Example%20Plugin%20%28Objective%2DC%29) for reference. | 4. **Implement your Frame Processing.** See the [Example Plugin (Objective-C)](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/ios/Frame%20Processor%20Plugins/Example%20Plugin/ExampleFrameProcessorPlugin.m) for reference. | ||||||
|  |  | ||||||
| </TabItem> | </TabItem> | ||||||
| <TabItem value="swift"> | <TabItem value="swift"> | ||||||
| @@ -117,7 +117,7 @@ VISION_EXPORT_SWIFT_FRAME_PROCESSOR(FaceDetectorFrameProcessorPlugin, detectFace | |||||||
| // highlight-end | // highlight-end | ||||||
| ``` | ``` | ||||||
|  |  | ||||||
| 5. **Implement your frame processing.** See [Example Plugin (Swift)](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/ios/Frame%20Processor%20Plugins/Example%20Plugin%20%28Swift%29) for reference. | 5. **Implement your frame processing.** See [Example Plugin (Swift)](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/ios/Frame%20Processor%20Plugins/Example%20Swift%20Plugin/ExampleSwiftFrameProcessor.swift) for reference. | ||||||
|  |  | ||||||
|  |  | ||||||
| </TabItem> | </TabItem> | ||||||
|   | |||||||
| @@ -13,7 +13,7 @@ import useBaseUrl from '@docusaurus/useBaseUrl' | |||||||
| ## Mocking VisionCamera | ## Mocking VisionCamera | ||||||
|  |  | ||||||
| These steps allow you to mock VisionCamera and use it for developing or testing. Based on | These steps allow you to mock VisionCamera and use it for developing or testing. Based on | ||||||
| [Detox Mock Guide](https://github.com/wix/Detox/blob/master/docs/Guide.Mocking.md). | [Detox Mock Guide](https://wix.github.io/Detox/docs/guide/mocking). | ||||||
|  |  | ||||||
| ### Configure the Metro bundler | ### Configure the Metro bundler | ||||||
|  |  | ||||||
|   | |||||||
| @@ -82,7 +82,7 @@ If you're experiencing build issues or runtime issues in VisionCamera, make sure | |||||||
|    yarn   # or `npm i` |    yarn   # or `npm i` | ||||||
|    ``` |    ``` | ||||||
| 4. Make sure you have installed the [Android NDK](https://developer.android.com/ndk). | 4. Make sure you have installed the [Android NDK](https://developer.android.com/ndk). | ||||||
| 5. Make sure your minimum SDK version is **26 or higher**, and target SDK version is **33 or higher**. See [the example's `build.gradle`](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/android/build.gradle#L5-L10) for reference. | 5. Make sure your minimum SDK version is **26 or higher**, and target SDK version is **33 or higher**. See [the example's `build.gradle`](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/android/build.gradle#L5-L9) for reference. | ||||||
|    1. Open your `build.gradle` |    1. Open your `build.gradle` | ||||||
|    2. Set `buildToolsVersion` to `33.0.0` or higher |    2. Set `buildToolsVersion` to `33.0.0` or higher | ||||||
|    3. Set `compileSdkVersion` to `33` or higher |    3. Set `compileSdkVersion` to `33` or higher | ||||||
|   | |||||||
| @@ -37,7 +37,7 @@ std::vector<jsi::PropNameID> VisionCameraProxy::getPropertyNames(jsi::Runtime& r | |||||||
|   std::vector<jsi::PropNameID> result; |   std::vector<jsi::PropNameID> result; | ||||||
|   result.push_back(jsi::PropNameID::forUtf8(runtime, std::string("setFrameProcessor"))); |   result.push_back(jsi::PropNameID::forUtf8(runtime, std::string("setFrameProcessor"))); | ||||||
|   result.push_back(jsi::PropNameID::forUtf8(runtime, std::string("removeFrameProcessor"))); |   result.push_back(jsi::PropNameID::forUtf8(runtime, std::string("removeFrameProcessor"))); | ||||||
|   result.push_back(jsi::PropNameID::forUtf8(runtime, std::string("getFrameProcessorPlugin"))); |   result.push_back(jsi::PropNameID::forUtf8(runtime, std::string("initFrameProcessorPlugin"))); | ||||||
|   return result; |   return result; | ||||||
| } | } | ||||||
|  |  | ||||||
| @@ -49,10 +49,10 @@ void VisionCameraProxy::removeFrameProcessor(int viewTag) { | |||||||
|   _javaProxy->cthis()->removeFrameProcessor(viewTag); |   _javaProxy->cthis()->removeFrameProcessor(viewTag); | ||||||
| } | } | ||||||
|  |  | ||||||
| jsi::Value VisionCameraProxy::getFrameProcessorPlugin(jsi::Runtime& runtime, const std::string& name, const jsi::Object& jsOptions) { | jsi::Value VisionCameraProxy::initFrameProcessorPlugin(jsi::Runtime& runtime, const std::string& name, const jsi::Object& jsOptions) { | ||||||
|   auto options = JSIJNIConversion::convertJSIObjectToJNIMap(runtime, jsOptions); |   auto options = JSIJNIConversion::convertJSIObjectToJNIMap(runtime, jsOptions); | ||||||
|  |  | ||||||
|   auto plugin = _javaProxy->cthis()->getFrameProcessorPlugin(name, options); |   auto plugin = _javaProxy->cthis()->initFrameProcessorPlugin(name, options); | ||||||
|  |  | ||||||
|   auto pluginHostObject = std::make_shared<FrameProcessorPluginHostObject>(plugin); |   auto pluginHostObject = std::make_shared<FrameProcessorPluginHostObject>(plugin); | ||||||
|   return jsi::Object::createFromHostObject(runtime, pluginHostObject); |   return jsi::Object::createFromHostObject(runtime, pluginHostObject); | ||||||
| @@ -80,9 +80,9 @@ jsi::Value VisionCameraProxy::get(jsi::Runtime& runtime, const jsi::PropNameID& | |||||||
|           return jsi::Value::undefined(); |           return jsi::Value::undefined(); | ||||||
|         }); |         }); | ||||||
|   } |   } | ||||||
|   if (name == "getFrameProcessorPlugin") { |   if (name == "initFrameProcessorPlugin") { | ||||||
|     return jsi::Function::createFromHostFunction( |     return jsi::Function::createFromHostFunction( | ||||||
|         runtime, jsi::PropNameID::forUtf8(runtime, "getFrameProcessorPlugin"), 1, |         runtime, jsi::PropNameID::forUtf8(runtime, "initFrameProcessorPlugin"), 1, | ||||||
|         [this](jsi::Runtime& runtime, const jsi::Value& thisValue, const jsi::Value* arguments, size_t count) -> jsi::Value { |         [this](jsi::Runtime& runtime, const jsi::Value& thisValue, const jsi::Value* arguments, size_t count) -> jsi::Value { | ||||||
|           if (count < 1 || !arguments[0].isString()) { |           if (count < 1 || !arguments[0].isString()) { | ||||||
|             throw jsi::JSError(runtime, "First argument needs to be a string (pluginName)!"); |             throw jsi::JSError(runtime, "First argument needs to be a string (pluginName)!"); | ||||||
| @@ -90,7 +90,7 @@ jsi::Value VisionCameraProxy::get(jsi::Runtime& runtime, const jsi::PropNameID& | |||||||
|           auto pluginName = arguments[0].asString(runtime).utf8(runtime); |           auto pluginName = arguments[0].asString(runtime).utf8(runtime); | ||||||
|           auto options = count > 1 ? arguments[1].asObject(runtime) : jsi::Object(runtime); |           auto options = count > 1 ? arguments[1].asObject(runtime) : jsi::Object(runtime); | ||||||
|  |  | ||||||
|           return this->getFrameProcessorPlugin(runtime, pluginName, options); |           return this->initFrameProcessorPlugin(runtime, pluginName, options); | ||||||
|         }); |         }); | ||||||
|   } |   } | ||||||
|  |  | ||||||
|   | |||||||
| @@ -28,7 +28,7 @@ public: | |||||||
| private: | private: | ||||||
|   void setFrameProcessor(int viewTag, jsi::Runtime& runtime, const jsi::Object& frameProcessor); |   void setFrameProcessor(int viewTag, jsi::Runtime& runtime, const jsi::Object& frameProcessor); | ||||||
|   void removeFrameProcessor(int viewTag); |   void removeFrameProcessor(int viewTag); | ||||||
|   jsi::Value getFrameProcessorPlugin(jsi::Runtime& runtime, const std::string& name, const jsi::Object& options); |   jsi::Value initFrameProcessorPlugin(jsi::Runtime& runtime, const std::string& name, const jsi::Object& options); | ||||||
|  |  | ||||||
| private: | private: | ||||||
|   jni::global_ref<JVisionCameraProxy::javaobject> _javaProxy; |   jni::global_ref<JVisionCameraProxy::javaobject> _javaProxy; | ||||||
|   | |||||||
| @@ -82,10 +82,10 @@ void JVisionCameraProxy::removeFrameProcessor(int viewTag) { | |||||||
|   removeFrameProcessorMethod(_javaPart, viewTag); |   removeFrameProcessorMethod(_javaPart, viewTag); | ||||||
| } | } | ||||||
|  |  | ||||||
| local_ref<JFrameProcessorPlugin::javaobject> JVisionCameraProxy::getFrameProcessorPlugin(const std::string& name, TOptions options) { | local_ref<JFrameProcessorPlugin::javaobject> JVisionCameraProxy::initFrameProcessorPlugin(const std::string& name, TOptions options) { | ||||||
|   auto getFrameProcessorPluginMethod = |   auto initFrameProcessorPluginMethod = | ||||||
|       javaClassLocal()->getMethod<JFrameProcessorPlugin(local_ref<jstring>, TOptions)>("getFrameProcessorPlugin"); |       javaClassLocal()->getMethod<JFrameProcessorPlugin(local_ref<jstring>, TOptions)>("initFrameProcessorPlugin"); | ||||||
|   return getFrameProcessorPluginMethod(_javaPart, make_jstring(name), std::move(options)); |   return initFrameProcessorPluginMethod(_javaPart, make_jstring(name), std::move(options)); | ||||||
| } | } | ||||||
|  |  | ||||||
| void JVisionCameraProxy::registerNatives() { | void JVisionCameraProxy::registerNatives() { | ||||||
|   | |||||||
| @@ -30,7 +30,7 @@ public: | |||||||
|  |  | ||||||
|   void setFrameProcessor(int viewTag, jsi::Runtime& runtime, const jsi::Object& frameProcessor); |   void setFrameProcessor(int viewTag, jsi::Runtime& runtime, const jsi::Object& frameProcessor); | ||||||
|   void removeFrameProcessor(int viewTag); |   void removeFrameProcessor(int viewTag); | ||||||
|   jni::local_ref<JFrameProcessorPlugin::javaobject> getFrameProcessorPlugin(const std::string& name, |   jni::local_ref<JFrameProcessorPlugin::javaobject> initFrameProcessorPlugin(const std::string& name, | ||||||
|                                                                              jni::local_ref<JMap<jstring, jobject>> options); |                                                                              jni::local_ref<JMap<jstring, jobject>> options); | ||||||
|  |  | ||||||
|   jsi::Runtime* getJSRuntime() { |   jsi::Runtime* getJSRuntime() { | ||||||
|   | |||||||
| @@ -7,24 +7,34 @@ import com.facebook.proguard.annotations.DoNotStrip; | |||||||
| import java.util.Map; | import java.util.Map; | ||||||
|  |  | ||||||
| /** | /** | ||||||
|  * Declares a Frame Processor Plugin. |  * The base class of a native Frame Processor Plugin. | ||||||
|  |  * | ||||||
|  |  * Subclass this to create a custom Frame Processor Plugin, which can be called from a JS Frame Processor. | ||||||
|  |  * Once subclassed, it needs to be registered in the VisionCamera Frame Processor | ||||||
|  |  * runtime via `FrameProcessorPluginRegistry.addFrameProcessorPlugin` - ideally at app startup. | ||||||
|  |  | ||||||
|  |  * See: <a href="https://react-native-vision-camera.com/docs/guides/frame-processors-plugins-android">Creating Frame Processor Plugins (Android)</a> | ||||||
|  |  * for more information | ||||||
|  */ |  */ | ||||||
| @DoNotStrip | @DoNotStrip | ||||||
| @Keep | @Keep | ||||||
| public abstract class FrameProcessorPlugin { | public abstract class FrameProcessorPlugin { | ||||||
|     public FrameProcessorPlugin() {} |  | ||||||
|  |  | ||||||
|     /** |     /** | ||||||
|      * The initializer for a Frame Processor Plugin class that takes optional object that consists |      * The initializer of this Frame Processor Plugin. | ||||||
|      * options passed from JS layer |      * This is called everytime this Frame Processor Plugin is loaded from the JS side (`initFrameProcessorPlugin(..)`). | ||||||
|  |      * Optionally override this method to implement custom initialization logic. | ||||||
|  |      * @param options An options dictionary passed from the JS side, or null if none. | ||||||
|      */ |      */ | ||||||
|     public FrameProcessorPlugin(@Nullable Map<String, Object> options) {} |     public FrameProcessorPlugin(@Nullable Map<String, Object> options) {} | ||||||
|  |  | ||||||
|     /** |     /** | ||||||
|      * The actual Frame Processor plugin callback. Called for every frame the ImageAnalyzer receives. |      * The actual Frame Processor Plugin's implementation that runs when `plugin.call(..)` is called in the JS Frame Processor. | ||||||
|  |      * Implement your Frame Processing here, and keep in mind that this is a hot-path so optimize as good as possible. | ||||||
|  |      * See: <a href="https://react-native-vision-camera.com/docs/guides/frame-processors-tips#fast-frame-processor-plugins">Performance Tips</a> | ||||||
|  |      * | ||||||
|      * @param frame The Frame from the Camera. Don't call .close() on this, as VisionCamera handles that. |      * @param frame The Frame from the Camera. Don't call .close() on this, as VisionCamera handles that. | ||||||
|      * @return You can return any primitive, map or array you want. See the |      * @return You can return any primitive, map or array you want. | ||||||
|      * <a href="https://react-native-vision-camera.com/docs/guides/frame-processors-plugins-overview#types">Types</a> |      *         See the <a href="https://react-native-vision-camera.com/docs/guides/frame-processors-plugins-overview#types">Types</a> | ||||||
|      *         table for a list of supported types. |      *         table for a list of supported types. | ||||||
|      */ |      */ | ||||||
|     @DoNotStrip |     @DoNotStrip | ||||||
|   | |||||||
| @@ -70,7 +70,7 @@ class VisionCameraProxy(context: ReactApplicationContext) { | |||||||
|  |  | ||||||
|   @DoNotStrip |   @DoNotStrip | ||||||
|   @Keep |   @Keep | ||||||
|   fun getFrameProcessorPlugin(name: String, options: Map<String, Any>): FrameProcessorPlugin = |   fun initFrameProcessorPlugin(name: String, options: Map<String, Any>): FrameProcessorPlugin = | ||||||
|     FrameProcessorPluginRegistry.getPlugin(name, options) |     FrameProcessorPluginRegistry.getPlugin(name, options) | ||||||
|  |  | ||||||
|   // private C++ funcs |   // private C++ funcs | ||||||
|   | |||||||
| @@ -41,6 +41,7 @@ public class ExampleFrameProcessorPlugin extends FrameProcessorPlugin { | |||||||
|     } |     } | ||||||
|  |  | ||||||
|     ExampleFrameProcessorPlugin(@Nullable Map<String, Object> options) { |     ExampleFrameProcessorPlugin(@Nullable Map<String, Object> options) { | ||||||
|         Log.d("ExamplePlugin", " - options: " + options.toString()); |         super(options); | ||||||
|  |         Log.d("ExamplePlugin", "ExampleFrameProcessorPlugin initialized with options: " + options); | ||||||
|     } |     } | ||||||
| } | } | ||||||
|   | |||||||
| @@ -6,7 +6,7 @@ import com.mrousavy.camera.frameprocessor.FrameProcessorPlugin | |||||||
|  |  | ||||||
| class ExampleKotlinFrameProcessorPlugin(options: Map<String, Any>?): FrameProcessorPlugin(options) { | class ExampleKotlinFrameProcessorPlugin(options: Map<String, Any>?): FrameProcessorPlugin(options) { | ||||||
|     init { |     init { | ||||||
|         Log.d("ExampleKotlinPlugin", " - options" + options?.toString()) |         Log.d("ExampleKotlinPlugin", "ExampleKotlinFrameProcessorPlugin initialized with options: " + options?.toString()) | ||||||
|     } |     } | ||||||
|  |  | ||||||
|     override fun callback(frame: Frame, params: Map<String, Any>?): Any? { |     override fun callback(frame: Frame, params: Map<String, Any>?): Any? { | ||||||
|   | |||||||
| @@ -21,7 +21,7 @@ | |||||||
| - (instancetype)initWithOptions:(NSDictionary* _Nullable)options | - (instancetype)initWithOptions:(NSDictionary* _Nullable)options | ||||||
| { | { | ||||||
|   self = [super initWithOptions:options]; |   self = [super initWithOptions:options]; | ||||||
|   NSLog(@"ExamplePlugin - options: %@", options); |   NSLog(@"ExampleFrameProcessorPlugin initialized with options: %@", options); | ||||||
|   return self; |   return self; | ||||||
| } | } | ||||||
|  |  | ||||||
|   | |||||||
| @@ -14,7 +14,7 @@ public class ExampleSwiftFrameProcessorPlugin: FrameProcessorPlugin { | |||||||
|   public override init(options: [AnyHashable: Any]! = [:]) { |   public override init(options: [AnyHashable: Any]! = [:]) { | ||||||
|     super.init(options: options) |     super.init(options: options) | ||||||
|  |  | ||||||
|     print("ExampleSwiftPlugin - options: \(String(describing: options))") |     print("ExampleSwiftFrameProcessorPlugin initialized with options: \(String(describing: options))") | ||||||
|   } |   } | ||||||
|  |  | ||||||
|   public override func callback(_ frame: Frame, withArguments arguments: [AnyHashable: Any]?) -> Any? { |   public override func callback(_ frame: Frame, withArguments arguments: [AnyHashable: Any]?) -> Any? { | ||||||
|   | |||||||
| @@ -1,6 +1,6 @@ | |||||||
| import { VisionCameraProxy, Frame } from 'react-native-vision-camera' | import { VisionCameraProxy, Frame } from 'react-native-vision-camera' | ||||||
|  |  | ||||||
| const plugin = VisionCameraProxy.getFrameProcessorPlugin('example_kotlin_swift_plugin', { foo: 'bar' }) | const plugin = VisionCameraProxy.initFrameProcessorPlugin('example_kotlin_swift_plugin', { foo: 'bar' }) | ||||||
|  |  | ||||||
| export function exampleKotlinSwiftPlugin(frame: Frame): string[] { | export function exampleKotlinSwiftPlugin(frame: Frame): string[] { | ||||||
|   'worklet' |   'worklet' | ||||||
|   | |||||||
| @@ -1,6 +1,6 @@ | |||||||
| import { VisionCameraProxy, Frame } from 'react-native-vision-camera' | import { VisionCameraProxy, Frame } from 'react-native-vision-camera' | ||||||
|  |  | ||||||
| const plugin = VisionCameraProxy.getFrameProcessorPlugin('example_plugin') | const plugin = VisionCameraProxy.initFrameProcessorPlugin('example_plugin') | ||||||
|  |  | ||||||
| interface Result { | interface Result { | ||||||
|   example_array: (string | number | boolean)[] |   example_array: (string | number | boolean)[] | ||||||
|   | |||||||
| @@ -11,22 +11,38 @@ | |||||||
| #import "Frame.h" | #import "Frame.h" | ||||||
| #import <Foundation/Foundation.h> | #import <Foundation/Foundation.h> | ||||||
|  |  | ||||||
| /// The base class for a Frame Processor Plugin which can be called synchronously from a JS Frame | /** | ||||||
| /// Processor. |  * The base class of a native Frame Processor Plugin. | ||||||
| /// |  * | ||||||
| /// Subclass this class in a Swift or Objective-C class and override the `callback:withArguments:` |  * Subclass this to create a custom Frame Processor Plugin, which can be called from a JS Frame Processor. | ||||||
| /// method, and implement your Frame Processing there. |  * Once subclassed, it needs to be registered in the VisionCamera Frame Processor runtime via | ||||||
| /// |  * the `VISION_EXPORT_FRAME_PROCESSOR` or `VISION_EXPORT_SWIFT_FRAME_PROCESSOR` macros. | ||||||
| /// Use `[FrameProcessorPluginRegistry addFrameProcessorPlugin:]` to register the Plugin to the |  | ||||||
| /// VisionCamera Runtime. |  * See: <a href="https://react-native-vision-camera.com/docs/guides/frame-processors-plugins-ios">Creating Frame Processor Plugins (iOS)</a> | ||||||
|  |  * for more information | ||||||
|  |  */ | ||||||
| @interface FrameProcessorPlugin : NSObject | @interface FrameProcessorPlugin : NSObject | ||||||
|  |  | ||||||
| /// The initializer for a Frame Processor Plugin class that takes optional object that consists | /** | ||||||
| /// options passed from JS layer |  * The initializer of this Frame Processor Plugin. | ||||||
|  |  * This is called everytime this Frame Processor Plugin is loaded from the JS side (`initFrameProcessorPlugin(..)`). | ||||||
|  |  * Optionally override this method to implement custom initialization logic. | ||||||
|  |  * - Parameters: | ||||||
|  |  *   - options: An options dictionary passed from the JS side, or `nil` if none. | ||||||
|  |  */ | ||||||
| - (instancetype _Nonnull)initWithOptions:(NSDictionary* _Nullable)options; | - (instancetype _Nonnull)initWithOptions:(NSDictionary* _Nullable)options; | ||||||
|  |  | ||||||
| /// The actual callback when calling this plugin. Any Frame Processing should be handled there. | /** | ||||||
| /// Make sure your code is optimized, as this is a hot path. |  * The actual Frame Processor Plugin's implementation that runs when `plugin.call(..)` is called in the JS Frame Processor. | ||||||
|  |  * Implement your Frame Processing here, and keep in mind that this is a hot-path so optimize as good as possible. | ||||||
|  |  * See: <a href="https://react-native-vision-camera.com/docs/guides/frame-processors-tips#fast-frame-processor-plugins">Performance Tips</a> | ||||||
|  |  * | ||||||
|  |  * - Parameters: | ||||||
|  |  *   - frame: The Frame from the Camera. Don't do any ref-counting on this, as VisionCamera handles that. | ||||||
|  |  * - Returns: You can return any primitive, map or array you want. | ||||||
|  |  *            See the <a href="https://react-native-vision-camera.com/docs/guides/frame-processors-plugins-overview#types">Types</a> | ||||||
|  |  *            table for a list of supported types. | ||||||
|  |  */ | ||||||
| - (id _Nullable)callback:(Frame* _Nonnull)frame withArguments:(NSDictionary* _Nullable)arguments; | - (id _Nullable)callback:(Frame* _Nonnull)frame withArguments:(NSDictionary* _Nullable)arguments; | ||||||
|  |  | ||||||
| @end | @end | ||||||
|   | |||||||
| @@ -30,7 +30,7 @@ public: | |||||||
| private: | private: | ||||||
|   void setFrameProcessor(jsi::Runtime& runtime, int viewTag, const jsi::Object& frameProcessor); |   void setFrameProcessor(jsi::Runtime& runtime, int viewTag, const jsi::Object& frameProcessor); | ||||||
|   void removeFrameProcessor(jsi::Runtime& runtime, int viewTag); |   void removeFrameProcessor(jsi::Runtime& runtime, int viewTag); | ||||||
|   jsi::Value getFrameProcessorPlugin(jsi::Runtime& runtime, std::string name, const jsi::Object& options); |   jsi::Value initFrameProcessorPlugin(jsi::Runtime& runtime, std::string name, const jsi::Object& options); | ||||||
|  |  | ||||||
| private: | private: | ||||||
|   std::shared_ptr<RNWorklet::JsiWorkletContext> _workletContext; |   std::shared_ptr<RNWorklet::JsiWorkletContext> _workletContext; | ||||||
|   | |||||||
| @@ -65,7 +65,7 @@ std::vector<jsi::PropNameID> VisionCameraProxy::getPropertyNames(jsi::Runtime& r | |||||||
|   std::vector<jsi::PropNameID> result; |   std::vector<jsi::PropNameID> result; | ||||||
|   result.push_back(jsi::PropNameID::forUtf8(runtime, std::string("setFrameProcessor"))); |   result.push_back(jsi::PropNameID::forUtf8(runtime, std::string("setFrameProcessor"))); | ||||||
|   result.push_back(jsi::PropNameID::forUtf8(runtime, std::string("removeFrameProcessor"))); |   result.push_back(jsi::PropNameID::forUtf8(runtime, std::string("removeFrameProcessor"))); | ||||||
|   result.push_back(jsi::PropNameID::forUtf8(runtime, std::string("getFrameProcessorPlugin"))); |   result.push_back(jsi::PropNameID::forUtf8(runtime, std::string("initFrameProcessorPlugin"))); | ||||||
|   return result; |   return result; | ||||||
| } | } | ||||||
|  |  | ||||||
| @@ -94,7 +94,7 @@ void VisionCameraProxy::removeFrameProcessor(jsi::Runtime& runtime, int viewTag) | |||||||
|   }); |   }); | ||||||
| } | } | ||||||
|  |  | ||||||
| jsi::Value VisionCameraProxy::getFrameProcessorPlugin(jsi::Runtime& runtime, std::string name, const jsi::Object& options) { | jsi::Value VisionCameraProxy::initFrameProcessorPlugin(jsi::Runtime& runtime, std::string name, const jsi::Object& options) { | ||||||
|   NSString* key = [NSString stringWithUTF8String:name.c_str()]; |   NSString* key = [NSString stringWithUTF8String:name.c_str()]; | ||||||
|   NSDictionary* optionsObjc = JSINSObjectConversion::convertJSIObjectToNSDictionary(runtime, options, _callInvoker); |   NSDictionary* optionsObjc = JSINSObjectConversion::convertJSIObjectToNSDictionary(runtime, options, _callInvoker); | ||||||
|   FrameProcessorPlugin* plugin = [FrameProcessorPluginRegistry getPlugin:key withOptions:optionsObjc]; |   FrameProcessorPlugin* plugin = [FrameProcessorPluginRegistry getPlugin:key withOptions:optionsObjc]; | ||||||
| @@ -128,9 +128,9 @@ jsi::Value VisionCameraProxy::get(jsi::Runtime& runtime, const jsi::PropNameID& | |||||||
|           return jsi::Value::undefined(); |           return jsi::Value::undefined(); | ||||||
|         }); |         }); | ||||||
|   } |   } | ||||||
|   if (name == "getFrameProcessorPlugin") { |   if (name == "initFrameProcessorPlugin") { | ||||||
|     return jsi::Function::createFromHostFunction( |     return jsi::Function::createFromHostFunction( | ||||||
|         runtime, jsi::PropNameID::forUtf8(runtime, "getFrameProcessorPlugin"), 1, |         runtime, jsi::PropNameID::forUtf8(runtime, "initFrameProcessorPlugin"), 1, | ||||||
|         [this](jsi::Runtime& runtime, const jsi::Value& thisValue, const jsi::Value* arguments, size_t count) -> jsi::Value { |         [this](jsi::Runtime& runtime, const jsi::Value& thisValue, const jsi::Value* arguments, size_t count) -> jsi::Value { | ||||||
|           if (count < 1 || !arguments[0].isString()) { |           if (count < 1 || !arguments[0].isString()) { | ||||||
|             throw jsi::JSError(runtime, "First argument needs to be a string (pluginName)!"); |             throw jsi::JSError(runtime, "First argument needs to be a string (pluginName)!"); | ||||||
| @@ -138,7 +138,7 @@ jsi::Value VisionCameraProxy::get(jsi::Runtime& runtime, const jsi::PropNameID& | |||||||
|           auto pluginName = arguments[0].asString(runtime).utf8(runtime); |           auto pluginName = arguments[0].asString(runtime).utf8(runtime); | ||||||
|           auto options = count > 1 ? arguments[1].asObject(runtime) : jsi::Object(runtime); |           auto options = count > 1 ? arguments[1].asObject(runtime) : jsi::Object(runtime); | ||||||
|  |  | ||||||
|           return this->getFrameProcessorPlugin(runtime, pluginName, options); |           return this->initFrameProcessorPlugin(runtime, pluginName, options); | ||||||
|         }); |         }); | ||||||
|   } |   } | ||||||
|  |  | ||||||
|   | |||||||
| @@ -24,10 +24,17 @@ interface TVisionCameraProxy { | |||||||
|   setFrameProcessor: (viewTag: number, frameProcessor: FrameProcessor) => void |   setFrameProcessor: (viewTag: number, frameProcessor: FrameProcessor) => void | ||||||
|   removeFrameProcessor: (viewTag: number) => void |   removeFrameProcessor: (viewTag: number) => void | ||||||
|   /** |   /** | ||||||
|    * Creates a new instance of a Frame Processor Plugin. |    * Creates a new instance of a native Frame Processor Plugin. | ||||||
|    * The Plugin has to be registered on the native side, otherwise this returns `undefined` |    * The Plugin has to be registered on the native side, otherwise this returns `undefined`. | ||||||
|  |    * @param name The name of the Frame Processor Plugin. This has to be the same name as on the native side. | ||||||
|  |    * @param options (optional) Options, as a native dictionary, passed to the constructor/init-function of the native plugin. | ||||||
|  |    * @example | ||||||
|  |    * ```ts | ||||||
|  |    * const plugin = VisionCameraProxy.initFrameProcessorPlugin('scanFaces', { model: 'fast' }) | ||||||
|  |    * if (plugin == null) throw new Error("Failed to load scanFaces plugin!") | ||||||
|  |    * ``` | ||||||
|    */ |    */ | ||||||
|   getFrameProcessorPlugin: (name: string, options?: Record<string, ParameterType>) => FrameProcessorPlugin | undefined |   initFrameProcessorPlugin: (name: string, options?: Record<string, ParameterType>) => FrameProcessorPlugin | undefined | ||||||
| } | } | ||||||
|  |  | ||||||
| let hasWorklets = false | let hasWorklets = false | ||||||
| @@ -66,7 +73,7 @@ try { | |||||||
| } | } | ||||||
|  |  | ||||||
| let proxy: TVisionCameraProxy = { | let proxy: TVisionCameraProxy = { | ||||||
|   getFrameProcessorPlugin: () => { |   initFrameProcessorPlugin: () => { | ||||||
|     throw new CameraRuntimeError('system/frame-processors-unavailable', 'Frame Processors are not enabled!') |     throw new CameraRuntimeError('system/frame-processors-unavailable', 'Frame Processors are not enabled!') | ||||||
|   }, |   }, | ||||||
|   removeFrameProcessor: () => { |   removeFrameProcessor: () => { | ||||||
|   | |||||||
		Reference in New Issue
	
	Block a user