Marc Rousavy
0c3cd66016
fix: Improve C++ safety by attaching Cache Invalidator to jsi::Runtime's lifecycle ( #1488 )
...
* fix: fix C++ lint
* fix: attach `InvalidateCacheOnDestroy` to `jsi::Runtime`
2023-02-21 15:44:43 +01:00
Marc Rousavy
12f850c8e1
feat: Draw onto Frame as if it was a Skia Canvas ( #1479 )
...
* Create Shaders.ts
* Add `previewType` and `enableFpsGraph`
* Add RN Skia native dependency
* Add Skia Preview View on iOS
* Pass 1
* Update FrameHostObject.mm
* Wrap Canvas
* Lockfiles
* fix: Fix stuff
* chore: Upgrade RNWorklets
* Add `previewType` to set the Preview
* feat: Add Example
* Update project.pbxproj
* `enableFpsGraph`
* Cache the `std::shared_ptr<FrameHostObject>`
* Update CameraView+RecordVideo.swift
* Update SkiaMetalCanvasProvider.mm
* Android: Integrate Skia Dependency
* fix: Use new Prefix
* Add example for rendering shader
* chore: Upgrade CameraX
* Remove KTX
* Enable `viewBinding`
* Revert "Enable `viewBinding`"
This reverts commit f2a603f53b33ea4311a296422ffd1a910ce03f9e.
* Revert "chore: Upgrade CameraX"
This reverts commit 8dc832cf8754490d31a6192e6c1a1f11cdcd94fe.
* Remove unneeded `ProcessCameraProvider.getInstance()` call
* fix: Add REA hotfix patch
* fix: Fix FrameHostObject dead in runAsync
* fix: Make `runAsync` run truly async by dropping new Frames while executing
* chore: Upgrade RN Worklets to latest
* chore: Upgrade RN Skia
* Revert "Remove KTX"
This reverts commit 253f586633f7af2da992d2279fc206dc62597129.
* Make Skia optional in CMake
* Fix import
* Update CMakeLists.txt
* Update build.gradle
* Update CameraView.kt
* Update CameraView.kt
* Update CameraView.kt
* Update Shaders.ts
* Center Blur
* chore: Upgrade RN Worklets
* feat: Add `toByteArray()`, `orientation`, `isMirrored` and `timestamp` to `Frame` (#1487 )
* feat: Implement `orientation` and `isMirrored` on Frame
* feat: Add `toArrayBuffer()` func
* perf: Do faster buffer copy
* feat: Implement `toArrayBuffer()` on Android
* feat: Add `orientation` and `isMirrored` to Android
* feat: Add `timestamp` to Frame
* Update Frame.ts
* Update JImageProxy.h
* Update FrameHostObject.cpp
* Update FrameHostObject.cpp
* Update CameraPage.tsx
* fix: Format Swift
2023-02-21 15:00:48 +01:00
Marc Rousavy
0635d4aba0
fix: Add missing <regex> header
2023-02-15 19:13:33 +01:00
Marc Rousavy
f0ea18115e
fix: Fix CI for V3 ( #1475 )
...
* fix: Fix CI for "Build Android"
* update versions
* Update Gemfile.lock
* format swift
* fix: Fix swift lint
* Update .swiftlint.yml
* Use C++17 for lint
* fix: Fix C++ lints
2023-02-15 17:24:33 +01:00
Marc Rousavy
30b56153db
feat: Sync Frame Processors (plus runAsync and runAtTargetFps) ( #1472 )
...
Before, Frame Processors ran on a separate Thread.
After, Frame Processors run fully synchronous and always at the same FPS as the Camera.
Two new functions have been introduced:
* `runAtTargetFps(fps: number, func: () => void)`: Runs the given code as often as the given `fps`, effectively throttling it's calls.
* `runAsync(frame: Frame, func: () => void)`: Runs the given function on a separate Thread for Frame Processing. A strong reference to the Frame is held as long as the function takes to execute.
You can use `runAtTargetFps` to throttle calls to a specific API (e.g. if your Camera is running at 60 FPS, but you only want to run face detection at ~25 FPS, use `runAtTargetFps(25, ...)`.)
You can use `runAsync` to run a heavy algorithm asynchronous, so that the Camera is not blocked while your algorithm runs. This is useful if your main sync processor draws something, and your async processor is doing some image analysis on the side.
You can also combine both functions.
Examples:
```js
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
console.log("I'm running at 60 FPS!")
}, [])
```
```js
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
console.log("I'm running at 60 FPS!")
runAtTargetFps(10, () => {
'worklet'
console.log("I'm running at 10 FPS!")
})
}, [])
```
```js
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
console.log("I'm running at 60 FPS!")
runAsync(frame, () => {
'worklet'
console.log("I'm running on another Thread, I can block for longer!")
})
}, [])
```
```js
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
console.log("I'm running at 60 FPS!")
runAtTargetFps(10, () => {
'worklet'
runAsync(frame, () => {
'worklet'
console.log("I'm running on another Thread at 10 FPS, I can block for longer!")
})
})
}, [])
```
2023-02-15 16:47:09 +01:00
Marc Rousavy
a0590dccb5
feat: Replace Reanimated with RN Worklets ( #1468 )
...
* Setup RN Worklets
* Use RN Worklets on iOS
* Fix console
* Add `installFrameProcessorBindings()` function
* Add `FrameProcessorPlugins` proxy (BREAKING CHANGE)
* Clean up docs
* Update FRAME_PROCESSORS.mdx
* Use RN Worklets 0.2.5
* feat: Android build setup
* Rewrite Android Frame Processor Part
* Update CMakeLists.txt
* fix: Add react-native-worklets Gradle dependency
* Update Podfile.lock
* fix build
* gradle:7.4.1
* Init JSI Bindings in method on Android
* Fix Folly flags
* fix: Init `FrameProcessorRuntimeManager` later
* fix: Wrap in `<GestureHandlerRootView>`
* Refactor plugins
* fix: Remove enableFrameProcessors
* Install RN Worklets from current GH master
* Update babel.config.js
* Update CameraViewModule.kt
* Update ImageProxyUtils.java
* feat: Upgrade to Reanimated v3
* fix: Fix crash on Worklet init
* Update RN Worklets to latest master
* fix: Simplify FP Plugins Proxy
2023-02-13 15:22:45 +01:00
Marc Rousavy
8bafd96c24
feat: Allow returning of ImageProxy in a Frame Processor ( #1149 )
...
* feat: Allow returning of ImageProxy in a Frame Processor
* chore: Clean up
* fix: Missing space
* Update useFrameProcessor.ts
* Revert "Update useFrameProcessor.ts"
This reverts commit 9c645489cdfdf2079972669756a2cd20cc81e25e.
2022-07-18 10:40:56 +02:00
Marc Rousavy
0904767cf2
fix: Log Stacktrace on Frame Processor Error ( #731 )
...
* fix: Log JS Stack on Error
* Android
* Format Stacktrace better
* Update FrameProcessorUtils.mm
* Allow unapproved C++11 headers
* Use `.c_str()`
2022-01-10 16:37:47 +01:00
Janic Duplessis
4584d33921
fix: Fix JNI crash in FrameHostObject dtor ( #689 )
2022-01-02 15:57:15 +01:00
Marc Rousavy
6025367c72
fix: Fix duplicate JHashMap definition by moving to custom namespace ( #610 )
...
Technically `JHashMap` is duplicated now, but in separate namespaces. If I were to remove my `JHashMap` (and `JArrayList`) definitions, the user is forced to use fbjni v3.
2021-11-23 15:48:07 +01:00
Marc Rousavy
c47639d86a
fix: Fix possible null-pointer error in convertJNIObjectToJSIValue
2021-10-14 12:46:42 +02:00
Marc Rousavy
42e791b4bd
perf: Remove FrameProcessorPlugin HybridClass ( #467 )
...
* Rename `JImageProxyHostObject` -> `FrameHostObject`
* `FrameProcessorPlugin` -> `JFrameProcessorPlugin` 1/2
* `FrameProcessorPlugin` -> `JFrameProcessorPlugin` 2/2
* Make `const`
* Make `getName()` instance based
* Update JFrameProcessorPlugin.h
* Update JImageProxy.h
* `T`
* T
* Remove default ctor
* Use `TSelf` again
* Return `local_ref<CameraView*>` instead of `CameraView*`
* Make `findCameraViewById` return a raw pointer again...
* Extract `setFrameProcessor` and `unsetFrameProcessor`
* Use `global_ref`
* Use `static_cast` for `FrameHostObject`
* Update FrameProcessorRuntimeManager.cpp
* Fix reference lint error
* linelength
* Fix `unsetFrameProcessor` call
2021-09-29 12:30:50 +02:00
Marc Rousavy
60ea779ffe
fix: De-allocate frame HybridClass with JNI class loader if using Hermes ( #455 )
...
* De-allocate `frame` HybridClass with JNI class loader if using Hermes
See 1b3a0c2612
2021-09-24 16:57:12 +02:00
Marc Rousavy
99fff446e5
fix: Fix calling multiple Plugins in a single Frame Processor ( #435 )
...
* fix: Fix JNI <-> JSI conversion for Integers
* Create another plugin and call them both serially
* Use inline formatter for `__android_log_write`
* Update FrameProcessorRuntimeManager.cpp
* Log plugin class type
* Use `pluginGlobal->cthis()`
* Log class name
* fix dumb error
* C++: Dynamically get JNI `javaPart_` class & method
* clean up PR
2021-09-24 10:19:30 +02:00
Marc Rousavy
c3cf75eb5f
fix: Fix JNI <-> JSI conversion for Integers
2021-09-15 10:14:52 +02:00
Marc Rousavy
c657bf8718
Revert "fix: Fix Frame Processor null-dereference error (use ref-counted JNI local_ref) ( #351 )"
...
This reverts commit d06bcf648c
2021-08-20 16:05:09 +02:00
Marc Rousavy
90f2a1ef7d
fix: Fix close being called on a collected reference (enforce alias_ref) ( #379 )
...
* fix: Fix `close` being called on a collected reference (enforce `alias_ref`)
* Remove `~FrameHostObject` on iOS too
2021-08-20 16:05:02 +02:00
Marc Rousavy
0d3f2cbd9d
fix: Fix crash when accessing planesCount or bytesPerRow ( #380 )
...
* fix: Fix `bytesPerRow` and `planeCount` crashing
* use direct call
* Call `getPlanesCount` and `getBytesPerRow` through java
* revert camerapage changes
* remove removed header import
* Update CameraPage.tsx
2021-08-20 16:02:34 +02:00
Marc Rousavy
d06bcf648c
fix: Fix Frame Processor null-dereference error (use ref-counted JNI local_ref) ( #351 )
2021-08-20 15:57:31 +02:00
Marc Rousavy
1c32726ef6
fix: Throw JS errors if JSI <-> JNI conversion failed ( #324 )
2021-08-05 10:10:49 +02:00
Marc Rousavy
b493576373
fix: Fix java.lang.NoSuchMethodError error for .toArrayList() ( #322 )
...
* fix: Fix `java.lang.NoSuchMethodError` error for `.toArrayList()`
* Fix `HashMap<K, V>` conversion to `jsi::Object`
* Update FRAME_PROCESSORS_CREATE_OVERVIEW.mdx
2021-08-04 16:07:08 +02:00
Marc Rousavy
0f7ee51333
feat: Add console logging support for Frame Processors ( #297 )
...
* Try to log to console via runOnJS
* Call `console.log`
* Create custom `VisionCameraScheduler`
* Fix scheduler call
* Call with this
* Fix console setting
* Move J---- to `java-bindings`
* c++ style
* Android: 1/2 Create custom Scheduler
* Android: 2/2 Use custom Scheduler
* Don't use `runOnJS`, use `__callAsync` directly
2021-07-30 10:27:45 +02:00
Marc Rousavy
123d0d9e9c
chore(deps): Upgrade react-native-reanimated to alpha.2 ( #309 )
...
* Upgrade react-native-reanimated to alpha.2
* fix Android header search path
* Make `const`
2021-07-30 09:50:09 +02:00
Marc Rousavy
4b4ea0ff33
fix: Fix UI Thread race condition in setFrameProcessor(...) ( #265 )
...
* fix: Fix UI Thread race condition in `setFrameProcessor(...)`
* Revert "fix: Fix UI Thread race condition in `setFrameProcessor(...)`"
This reverts commit 9c524e123cff6843d7d11db602a5027d1bb06b4b.
* Use `setImmediate` to call `setFrameProcessor(...)`
* Fix frame processor order of applying
* Add `enableFrameProcessor` prop that defines if a FP is added
* rename constant
* Implement `enableFrameProcessor` prop for Android and make `frameProcessorFps` faster
* link to troubleshooting guide
* Update TROUBLESHOOTING.mdx
* Add logs for use-cases
* fix log
* set initial frame processor in `onLayout` instead of `componentDidMount`
2021-07-12 15:16:03 +02:00
Marc Rousavy
1e64215d37
chore(deps): upgrade camerax ( #257 )
...
* Upgrade CameraX to alpha6
* Upgrade CameraX extensions to alpha26
* `init` -> `getInstance`
* Use new Extensions API
* Update CameraView.kt
* use new ExtensionsManager API in `getAvailableCameraDevices()`
* fix cpplint errors
2021-07-07 12:57:28 +02:00
Marc Rousavy
2fa0f8fd46
chore: Clean up root directory ( #236 )
...
* Clean up root
* remove unused REA patches
* Remove `cpp/` and create headers per platform
* fix `#import`
* fix `VISION_CAMERA_DISABLE_FRAME_PROCESSORS`
2021-07-06 16:42:58 +02:00
Marc Rousavy
fa5f5c0cab
feat: Implement Frame.close() ( #229 )
...
* Implement `Frame.close()`
* close frame in dtor
* Update JImageProxyHostObject.cpp
* fix close
* Check if closed
* remove a few logs
* r
* fix `isValid` and `isReady`
* Add JImage
* Release JNI frame ref on destroy
* fix pod setup
* Fix isValid call
* Fix `close` not returning a function
* throw error if closed twice
* iOS: Schedule `console.error` call on JS thread
* Android: Log Frame Processor Error to JS
* fix syntax
* Check if valid `toString()`
* Update Frame.ts
* Remove `isReady`
* Fix JImage accessors
* remove `JImage` C++ sources
* Throw error if accessing props on closed Frame
* Delete `JImage.h`
2021-07-06 10:08:44 +02:00
Marc Rousavy
bfb1b67c2f
Update FrameProcessorRuntimeManager.cpp
2021-06-29 10:38:35 +02:00
Marc Rousavy
5ab1a72e01
chore: Fix C++ indents
2021-06-29 10:38:13 +02:00
Marc Rousavy
da84a0fd42
perf: Avoid error message copying
2021-06-28 20:45:08 +02:00
Marc Rousavy
4bacee796a
feat: Print Frame Processor errors to Metro console
2021-06-28 20:44:50 +02:00
Marc Rousavy
26d19d5152
perf: Remove a few unnecessary log statements
2021-06-28 18:30:42 +02:00
Marc Rousavy
e13bef7d17
fix: Host Object error
2021-06-28 10:19:19 +02:00
Marc Rousavy
87e6bb710e
feat: Frame Processors for Android ( #196 )
...
* Create android gradle build setup
* Fix `prefab` config
* Add `pickFirst **/*.so` to example build.gradle
* fix REA path
* cache gradle builds
* Update validate-android.yml
* Create Native Proxy
* Copy REA header
* implement ctor
* Rename CameraViewModule -> FrameProcessorRuntimeManager
* init FrameProcessorRuntimeManager
* fix name
* Update FrameProcessorRuntimeManager.h
* format
* Create AndroidErrorHandler.h
* Initialize runtime and install JSI funcs
* Update FrameProcessorRuntimeManager.cpp
* Update CameraViewModule.kt
* Make CameraView hybrid C++ class to find view & set frame processor
* Update FrameProcessorRuntimeManager.cpp
* pass function by rvalue
* pass by const &&
* extract hermes and JSC REA
* pass `FOR_HERMES`
* correctly prepare JSC and Hermes
* Update CMakeLists.txt
* add missing hermes include
* clean up imports
* Create JImageProxy.h
* pass ImageProxy to JNI as `jobject`
* try use `JImageProxy` C++ wrapper type
* Use `local_ref<JImageProxy>`
* Create `JImageProxyHostObject` for JSI interop
* debug call to frame processor
* Unset frame processor
* Fix CameraView native part not being registered
* close image
* use `jobject` instead of `JImageProxy` for now :(
* fix hermes build error
* Set enable FP callback
* fix JNI call
* Update CameraView.cpp
* Get Format
* Create plugin abstract
* Make `FrameProcessorPlugin` a hybrid object
* Register plugin CXX
* Call `registerPlugin`
* Catch
* remove JSI
* Create sample QR code plugin
* register plugins
* Fix missing JNI binding
* Add `mHybridData`
* prefix name with two underscores (`__`)
* Update CameraPage.tsx
* wrap `ImageProxy` in host object
* Use `jobject` for HO box
* Update JImageProxy.h
* reinterpret jobject
* Try using `JImageProxy` instead of `jobject`
* Update JImageProxy.h
* get bytes per row and plane count
* Update CameraView.cpp
* Return base
* add some docs and JNI JSI conversion
* indent
* Convert JSI value to JNI jobject
* using namespace facebook
* Try using class
* Use plain old Object[]
* Try convert JNI -> JSI
* fix decl
* fix bool init
* Correctly link folly
* Update CMakeLists.txt
* Convert Map to Object
* Use folly for Map and Array
* Return `alias_ref<jobject>` instead of raw `jobject`
* fix JNI <-> JSI conversion
* Update JSIJNIConversion.cpp
* Log parameters
* fix params index offset
* add more test cases
* Update FRAME_PROCESSORS_CREATE_OVERVIEW.mdx
* fix types
* Rename to example plugin
* remove support for hashmap
* Try use HashMap iterable fbjni binding
* try using JReadableArray/JReadableMap
* Fix list return values
* Update JSIJNIConversion.cpp
* Update JSIJNIConversion.cpp
* (iOS) Rename ObjC QR Code Plugin to Example Plugin
* Rename Swift plugin QR -> Example
* Update ExamplePluginSwift.swift
* Fix Map/Dictionary logging format
* Update ExampleFrameProcessorPlugin.m
* Reconfigure session if frame processor changed
* Handle use-cases via `maxUseCasesCount`
* Don't crash app on `configureSession` error
* Document "use-cases"
* Update DEVICES.mdx
* fix merge
* Make `const &`
* iOS: Automatically enable `video` if a `frameProcessor` is set
* Update CameraView.cpp
* fix docs
* Automatically fallback to snapshot capture if `supportsParallelVideoProcessing` is false.
* Fix lookup
* Update CameraView.kt
* Implement `frameProcessorFps`
* Finalize Frame Processor Plugin Hybrid
* Update CameraViewModule.kt
* Support `flash` on `takeSnapshot()`
* Update docs
* Add docs
* Update CameraPage.tsx
* Attribute NonNull
* remove unused imports
* Add Android docs for Frame Processors
* Make JNI HashMap <-> JSI Object conversion faster
directly access `toHashMap` instead of going through java
* add todo
* Always run `prepareJSC` and `prepareHermes`
* switch jsc and hermes
* Specify ndkVersion `21.4.7075529`
* Update gradle.properties
* Update gradle.properties
* Create .aar
* Correctly prepare android package
* Update package.json
* Update package.json
* remove `prefab` build feature
* split
* Add docs for registering the FP plugin
* Add step for dep
* Update CaptureButton.tsx
* Move to `reanimated-headers/`
* Exclude reanimated-headers from cpplint
* disable `build/include_order` rule
* cpplint fixes
* perf: Make `JSIJNIConversion` a `namespace` instead of `class`
* Ignore runtime/references for `convert` funcs
* Build Android .aar in CI
* Run android build script only on `prepack`
* Update package.json
* Update package.json
* Update build-android-npm-package.sh
* Move to `yarn build`
* Also install node_modules in example step
* Update validate-android.yml
* sort imports
* fix torch
* Run ImageAnalysis on `FrameProcessorThread`
* Update Errors.kt
* Add clean android script
* Upgrade reanimated to 2.3.0-alpha.1
* Revert "Upgrade reanimated to 2.3.0-alpha.1"
This reverts commit c1d3bed5e03728d0b5e335a359524ff4f56f5035.
* ⚠️ TEMP FIX: hotfix reanimated build.gradle
* Update CameraView+TakeSnapshot.kt
* ⚠️ TEMP FIX: Disable ktlint action for now
* Update clean.sh
* Set max heap size to 4g
* rebuild lockfiles
* Update Podfile.lock
* rename
* Build lib .aar before example/
2021-06-27 12:37:54 +02:00
