railbird/react-native-vision-camera
9
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
973 Commits 15 Branches 0 Tags 57 MiB
78a64abb8c
Commit Graph

201 Commits

Author SHA1 Message Date
Marc Rousavy
78a64abb8c chore: Bump version 2023-07-31 11:31:55 +02:00
Marc Rousavy
86dd703c2b
feat: Rewrite Android C++ part (VisionCameraProxy + JFrame) (#1661) 
* First Android rewrite

* Rewrite Android C++ backend

* Pass `ReadableNativeMap`, fix build error

* fix: Fix FrameProcessor init

* Make a bunch of stuff const reference to avoid copies

* Indents

* Cleanup

* indents

* docs: Update Android docs

* Update CameraView.kt

* fix: Format C++ code
 2023-07-22 00:15:11 +02:00
Marc Rousavy
44ed42d5d6
feat: Expose unified VisionCameraProxy object, make FrameProcessorPlugins object-oriented (#1660) 
* feat: Replace `FrameProcessorRuntimeManager` with `VisionCameraProxy` (iOS)

* Make `FrameProcessorPlugin` a constructable HostObject

* fix: Fix `name` override

* Simplify `useFrameProcessor

* fix: Fix lint errors

* Remove FrameProcessorPlugin::name

* JSIUtils -> JSINSObjectConversion
 2023-07-21 17:52:30 +02:00
Marc Rousavy
375e894038
feat: Complete iOS Codebase rewrite (#1647) 
* Make Frame Processors an extra subspec

* Update VisionCamera.podspec

* Make optional

* Make VisionCamera compile without Skia

* Fix

* Add skia again

* Update VisionCamera.podspec

* Make VisionCamera build without Frame Processors

* Rename error to `system/frame-processors-unavailable`

* Fix Frame Processor returning early

* Remove `preset`, FP partial rewrite

* Only warn on frame drop

* Fix wrong queue

* fix: Run on CameraQueue again

* Update CameraView.swift

* fix: Activate audio session asynchronously on audio queue

* Update CameraView+RecordVideo.swift

* Update PreviewView.h

* Cleanups

* Cleanup

* fix cast

* feat: Add LiDAR Depth Camera support

* Upgrade Ruby

* Add vector icons type

* Update Gemfile.lock

* fix: Stop queues on deinit

* Also load `builtInTrueDepthCamera`

* Update CameraViewManager.swift

* Update SkImageHelpers.mm

* Extract FrameProcessorCallback to FrameProcessor

Holds more context now :)

* Rename to .m

* fix: Add `RCTLog` import

* Create SkiaFrameProcessor

* Update CameraBridge.h

* Call Frame Processor

* Fix defines

* fix: Allow deleting callback funcs

* fix Skia build

* batch

* Just call `setSkiaFrameProcessor`

* Rewrite in Swift

* Pass `SkiaRenderer`

* Fix Import

* Move `PreviewView` to Swift

* Fix Layer

* Set Skia Canvas to Frame Host Object

* Make `DrawableFrameHostObject` subclass

* Fix TS types

* Use same MTLDevice and apply scale

* Make getter

* Extract `setTorch` and `Preview`

* fix: Fix nil metal device

* Don't wait for session stop in deinit

* Use main pixel ratio

* Use unique_ptr for Render Contexts

* fix: Fix SkiaPreviewDisplayLink broken after deinit

* inline `getTextureCache`

* Update CameraPage.tsx

* chore: Format iOS

* perf: Allow MTLLayer to be optimized for only frame buffers

* Add RN Video types

* fix: Fix Frame Processors if guard

* Find nodeModules recursively

* Create `Frame.isDrawable`

* Add `cocoapods-check` dependency
 2023-07-20 15:30:04 +02:00
Marc Rousavy
0139324da8 fix: Use correct Gradle version 2023-07-03 22:21:02 +02:00
Marc Rousavy
66c64814cc fix: Downgrade Gradle to 7.5.1 2023-07-03 22:15:06 +02:00
Marc Rousavy
cde8b762a3 chore: Remove react-native-slider 2023-07-03 22:10:20 +02:00
Marc Rousavy
7ae15af8cd feat: Add RN 0.72 support (namespace in build.gradle) 2023-07-03 22:07:49 +02:00
Marc Rousavy
0111f2edb4 fix: Simplify Podfile 2023-07-03 13:37:43 +02:00
Marc Rousavy
4ad4804e51
chore: Upgrade Skia to new Ganesh Rendering Pipeline and RN to 0.72 (#1638) 
* chore: Upgrade all Deps RN 72

* fix breaking skia changes

* fix devDeps

* Update yarn.lock

* fix metro config

* Update yarn.lock
 2023-07-03 12:40:07 +02:00
Marc Rousavy
f03dde725e
fix: Fix Skia build (#1582) 
* Add Skia search path from `node_modules/`

* fix: Find `react-native-skia` from node_modules

* fix: Upgrade RN Skia to use includes fix

* Update FrameProcessorUtils.mm
 2023-05-03 14:27:34 +02:00
Marc Rousavy
032798331a chore: Upgrade RN Skia 2023-05-02 12:54:18 +02:00
Marc Rousavy
bf52e8dad7 chore: Upgrade React Native for Xcode fix 2023-04-24 22:40:14 +02:00
Marc Rousavy
bb7ab30a0e fix: Set min iOS version to 12.4 2023-04-24 22:32:20 +02:00
Marc Rousavy
e1973b9b8d chore: Use Reanimated 3 from npm 2023-03-21 15:19:32 +01:00
Marc Rousavy
622d3830f1
feat: Make Frame Processor Plugins object-oriented on iOS as well (#1496) 
* feat: Make Frame Processor Plugins object-oriented on iOS as well

* Add Plugin in AppDelegate
 2023-02-27 11:18:03 +01:00
Marc Rousavy
12f850c8e1
feat: Draw onto Frame as if it was a Skia Canvas (#1479) 
* Create Shaders.ts

* Add `previewType` and `enableFpsGraph`

* Add RN Skia native dependency

* Add Skia Preview View on iOS

* Pass 1

* Update FrameHostObject.mm

* Wrap Canvas

* Lockfiles

* fix: Fix stuff

* chore: Upgrade RNWorklets

* Add `previewType` to set the Preview

* feat: Add Example

* Update project.pbxproj

* `enableFpsGraph`

* Cache the `std::shared_ptr<FrameHostObject>`

* Update CameraView+RecordVideo.swift

* Update SkiaMetalCanvasProvider.mm

* Android: Integrate Skia Dependency

* fix: Use new Prefix

* Add example for rendering shader

* chore: Upgrade CameraX

* Remove KTX

* Enable `viewBinding`

* Revert "Enable `viewBinding`"

This reverts commit f2a603f53b33ea4311a296422ffd1a910ce03f9e.

* Revert "chore: Upgrade CameraX"

This reverts commit 8dc832cf8754490d31a6192e6c1a1f11cdcd94fe.

* Remove unneeded `ProcessCameraProvider.getInstance()` call

* fix: Add REA hotfix patch

* fix: Fix FrameHostObject dead in runAsync

* fix: Make `runAsync` run truly async by dropping new Frames while executing

* chore: Upgrade RN Worklets to latest

* chore: Upgrade RN Skia

* Revert "Remove KTX"

This reverts commit 253f586633f7af2da992d2279fc206dc62597129.

* Make Skia optional in CMake

* Fix import

* Update CMakeLists.txt

* Update build.gradle

* Update CameraView.kt

* Update CameraView.kt

* Update CameraView.kt

* Update Shaders.ts

* Center Blur

* chore: Upgrade RN Worklets

* feat: Add `toByteArray()`, `orientation`, `isMirrored` and `timestamp` to `Frame` (#1487)

* feat: Implement `orientation` and `isMirrored` on Frame

* feat: Add `toArrayBuffer()` func

* perf: Do faster buffer copy

* feat: Implement `toArrayBuffer()` on Android

* feat: Add `orientation` and `isMirrored` to Android

* feat: Add `timestamp` to Frame

* Update Frame.ts

* Update JImageProxy.h

* Update FrameHostObject.cpp

* Update FrameHostObject.cpp

* Update CameraPage.tsx

* fix: Format Swift
 2023-02-21 15:00:48 +01:00
Marc Rousavy
f0ea18115e
fix: Fix CI for V3 (#1475) 
* fix: Fix CI for "Build Android"

* update versions

* Update Gemfile.lock

* format swift

* fix: Fix swift lint

* Update .swiftlint.yml

* Use C++17 for lint

* fix: Fix C++ lints
 2023-02-15 17:24:33 +01:00
Marc Rousavy
30b56153db
feat: Sync Frame Processors (plus runAsync and runAtTargetFps) (#1472) 
Before, Frame Processors ran on a separate Thread.

After, Frame Processors run fully synchronous and always at the same FPS as the Camera.

Two new functions have been introduced:

* `runAtTargetFps(fps: number, func: () => void)`: Runs the given code as often as the given `fps`, effectively throttling it's calls.
* `runAsync(frame: Frame, func: () => void)`: Runs the given function on a separate Thread for Frame Processing. A strong reference to the Frame is held as long as the function takes to execute.

You can use `runAtTargetFps` to throttle calls to a specific API (e.g. if your Camera is running at 60 FPS, but you only want to run face detection at ~25 FPS, use `runAtTargetFps(25, ...)`.)

You can use `runAsync` to run a heavy algorithm asynchronous, so that the Camera is not blocked while your algorithm runs. This is useful if your main sync processor draws something, and your async processor is doing some image analysis on the side. 

You can also combine both functions.

Examples:

```js
const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  console.log("I'm running at 60 FPS!")
}, [])
```

```js
const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  console.log("I'm running at 60 FPS!")

  runAtTargetFps(10, () => {
    'worklet'
    console.log("I'm running at 10 FPS!")
  })
}, [])
```



```js
const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  console.log("I'm running at 60 FPS!")

  runAsync(frame, () => {
    'worklet'
    console.log("I'm running on another Thread, I can block for longer!")
  })
}, [])
```

```js
const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  console.log("I'm running at 60 FPS!")

  runAtTargetFps(10, () => {
    'worklet'
    runAsync(frame, () => {
      'worklet'
      console.log("I'm running on another Thread at 10 FPS, I can block for longer!")
    })
  })
}, [])
```
 2023-02-15 16:47:09 +01:00
Marc Rousavy
a0590dccb5
feat: Replace Reanimated with RN Worklets (#1468) 
* Setup RN Worklets

* Use RN Worklets on iOS

* Fix console

* Add `installFrameProcessorBindings()` function

* Add `FrameProcessorPlugins` proxy (BREAKING CHANGE)

* Clean up docs

* Update FRAME_PROCESSORS.mdx

* Use RN Worklets 0.2.5

* feat: Android build setup

* Rewrite Android Frame Processor Part

* Update CMakeLists.txt

* fix: Add react-native-worklets Gradle dependency

* Update Podfile.lock

* fix build

* gradle:7.4.1

* Init JSI Bindings in method on Android

* Fix Folly flags

* fix: Init `FrameProcessorRuntimeManager` later

* fix: Wrap in `<GestureHandlerRootView>`

* Refactor plugins

* fix: Remove enableFrameProcessors

* Install RN Worklets from current GH master

* Update babel.config.js

* Update CameraViewModule.kt

* Update ImageProxyUtils.java

* feat: Upgrade to Reanimated v3

* fix: Fix crash on Worklet init

* Update RN Worklets to latest master

* fix: Simplify FP Plugins Proxy
 2023-02-13 15:22:45 +01:00
Marc Rousavy
11d1e7178d
chore: Upgrade to RN 71 (#1465) 
* chore: Upgrade Example to RN 0.71

* chore: Upgrade all libs

* fix: Fix CameraRoll installation

* Update Gradle Tools

* fix: Fix buildscripts

* Clean out build.gradle

* fix: Fix Kotlin setup

* fix: Move kotlin-android dependency to lib

* Move `_setGlobalConsole`

* Update gradle-wrapper.properties

* Rebuild lockfiles

* chore: Update build:gradle

* Update StatusBarBlurBackground.tsx

* Use Java 11 in Workflows

* Update MediaPage.tsx

* Add `google` repository to build.gradle

* Double Java Heap size

* Increase heap size

* Alternative args

* Update build.gradle
 2023-02-09 11:52:41 +01:00
Marc Rousavy
231838772c Update Podfile.lock 2022-11-30 12:06:11 +01:00
Thomas Coldwell
52a1d50d91
fix: Frame Processor FPS (#1288) 
* fix: Build using XCode 14

* fix: Throttle FP by start time rather than end time
 2022-10-20 12:49:22 +02:00
Marc Rousavy
8bafd96c24
feat: Allow returning of ImageProxy in a Frame Processor (#1149) 
* feat: Allow returning of ImageProxy in a Frame Processor

* chore: Clean up

* fix: Missing space

* Update useFrameProcessor.ts

* Revert "Update useFrameProcessor.ts"

This reverts commit 9c645489cdfdf2079972669756a2cd20cc81e25e.
 2022-07-18 10:40:56 +02:00
Thomas Coldwell
fb2156ec39
fix: Asset Writer Video-Audio Sync (#1075) 
* fix: Start asset writer session on first frame

* fix: Remove debug log

* fix: Reset dev team
 2022-06-11 11:15:24 +02:00
Marc Rousavy
b873867421 Update Podfile.lock 2022-03-30 10:33:38 +02:00
Marc Rousavy
cae1efe87d chore: Update lockfiles 2022-03-22 11:14:40 +01:00
Marc Rousavy
17b33de4d7 fix: Simpler goBack 2022-01-10 17:13:20 +01:00
Marc Rousavy
137f5272e3 fix: Fix HDR icon not showing 2022-01-10 17:10:29 +01:00
Antonio Sousa
233094d18e
chore: fix typo in README.md (#718) 2022-01-05 14:51:44 +01:00
Marc Rousavy
48da1819fc
feat: Custom Orientation (#715) 
* feat: Custom Orientation

* Update CameraView.swift

* Update CameraView.swift

* Try outputRotation approach

* whoops

* fix: Refactor `VideoCapture` instance

* Update orientation in didSetProps

* Update Orientation in iOS

* expose to objc

* Fix Orientation values

* format
 2022-01-04 16:57:40 +01:00
Marc Rousavy
dbfdf82c67 Update Podfile.lock 2022-01-04 09:19:21 +01:00
Marc Rousavy
7460560323 Update Podfile.lock 2021-12-30 10:43:33 +01:00
Marc Rousavy
1f4d188c52 fix: Fix Example app's exported property 2021-10-13 10:31:22 +02:00
Marc Rousavy
1cf4f110f4 chore: Rebuild lockfiles 2021-10-11 18:34:42 +02:00
dependabot[bot]
f57daf2177
chore(deps): bump @react-native-community/slider in /example (#512) 
Bumps [@react-native-community/slider](https://github.com/callstack/react-native-slider) from 4.1.7 to 4.1.8.
- [Release notes](https://github.com/callstack/react-native-slider/releases)
- [Commits](https://github.com/callstack/react-native-slider/compare/v4.1.7...v4.1.8)

---
updated-dependencies:
- dependency-name: "@react-native-community/slider"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-11 18:33:17 +02:00
dependabot[bot]
8652c6e157
chore(deps): bump @react-navigation/native-stack in /example (#505) 
Bumps [@react-navigation/native-stack](https://github.com/react-navigation/react-navigation/tree/HEAD/packages/native-stack) from 6.2.2 to 6.2.3.
- [Release notes](https://github.com/react-navigation/react-navigation/releases)
- [Changelog](https://github.com/react-navigation/react-navigation/blob/main/packages/native-stack/CHANGELOG.md)
- [Commits](https://github.com/react-navigation/react-navigation/commits/@react-navigation/native-stack@6.2.3/packages/native-stack)

---
updated-dependencies:
- dependency-name: "@react-navigation/native-stack"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-11 18:30:58 +02:00
dependabot[bot]
7e426a1244
chore(deps-dev): bump @babel/core from 7.15.5 to 7.15.8 in /example (#507) 
Bumps [@babel/core](https://github.com/babel/babel/tree/HEAD/packages/babel-core) from 7.15.5 to 7.15.8.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.15.8/packages/babel-core)

---
updated-dependencies:
- dependency-name: "@babel/core"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-11 18:29:27 +02:00
dependabot[bot]
0fe3ca9691
chore(deps): bump @react-native-community/cameraroll in /example (#508) 
Bumps [@react-native-community/cameraroll](https://github.com/react-native-community/react-native-cameraroll) from 4.0.4 to 4.1.1.
- [Release notes](https://github.com/react-native-community/react-native-cameraroll/releases)
- [Changelog](https://github.com/react-native-cameraroll/react-native-cameraroll/blob/master/.releaserc)
- [Commits](https://github.com/react-native-community/react-native-cameraroll/compare/v4.0.4...v4.1.1)

---
updated-dependencies:
- dependency-name: "@react-native-community/cameraroll"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-11 18:29:12 +02:00
dependabot[bot]
565706b8ca
chore(deps): bump @react-navigation/native in /example (#510) 
Bumps [@react-navigation/native](https://github.com/react-navigation/react-navigation/tree/HEAD/packages/native) from 6.0.4 to 6.0.5.
- [Release notes](https://github.com/react-navigation/react-navigation/releases)
- [Changelog](https://github.com/react-navigation/react-navigation/blob/main/packages/native/CHANGELOG.md)
- [Commits](https://github.com/react-navigation/react-navigation/commits/@react-navigation/native@6.0.5/packages/native)

---
updated-dependencies:
- dependency-name: "@react-navigation/native"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-11 18:28:43 +02:00
Marc Rousavy
2cf8087ad6 Update Podfile.lock 2021-10-10 15:57:22 +02:00
Marc Rousavy
8d24e344c9
chore: Upgrade CameraX to alpha09 (#472) 
* chore: Upgrade CameraX to alpha09

* Remove custom ProGuard file

It's no longer needed, CameraX now ships one.

* Set `targetSdkVersion` to `31`

* set `compileSdkVersion` to 31

* Add `android:exported=false`
 2021-10-07 12:20:21 +02:00
Marc Rousavy
bad9f99ac7
Update Info.plist 2021-10-07 10:30:31 +02:00
Marc Rousavy
916278d3ea
feat: Add React Native 0.66 support (#490) 
* feat: Add React Native 0.66 support

* Generate lockfiles

* Update Podfile.lock

* chore: Migrate from react-native-navigation to react-navigation (#491)

* Migrate RNN -> RN

* Migrate all screens

* Fix get permission status

* fix app name

* Update AppDelegate.m

* Fix Info.plist

* Set `UIViewControllerBasedStatusBarAppearance` to `YES`

* Only enable `audio` if user granted microphone permission

* Update App.tsx

* Fix RNGH for Android

* Use `navigate` instead of `push`

* Fix animation

* Upgrade @types/react-native

* "Splash" -> "PermissionsPage"
 2021-10-05 12:22:14 +02:00
dependabot[bot]
45c37c87dd
chore(deps): bump react-native-screens from 3.7.2 to 3.8.0 in /example (#479) 
Bumps [react-native-screens](https://github.com/software-mansion/react-native-screens) from 3.7.2 to 3.8.0.
- [Release notes](https://github.com/software-mansion/react-native-screens/releases)
- [Commits](https://github.com/software-mansion/react-native-screens/compare/3.7.2...3.8.0)

---
updated-dependencies:
- dependency-name: react-native-screens
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-05 10:39:23 +02:00
dependabot[bot]
979a72decc
chore(deps-dev): bump @types/react from 17.0.24 to 17.0.27 in /example (#477) 
Bumps [@types/react](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/react) from 17.0.24 to 17.0.27.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/react)

---
updated-dependencies:
- dependency-name: "@types/react"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-05 10:39:13 +02:00
dependabot[bot]
1714be8c05
chore(deps-dev): bump @types/react-native-vector-icons in /example (#463) 
Bumps [@types/react-native-vector-icons](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/react-native-vector-icons) from 6.4.8 to 6.4.9.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/react-native-vector-icons)

---
updated-dependencies:
- dependency-name: "@types/react-native-vector-icons"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-29 12:31:55 +02:00
Marc Rousavy
60ea779ffe
fix: De-allocate frame HybridClass with JNI class loader if using Hermes (#455) 
* De-allocate `frame` HybridClass with JNI class loader if using Hermes

See 1b3a0c2612

* Don't wrap in `#if FOR_HERMES`, other `jsi::Runtime`s might also run GC on another Thread.

* Use `jni::local_ref` for `FrameHostObject`

* Update JImageProxyHostObject.cpp

* Only run with JNI `ClassLoader` if ctor Thread ID != dtor Thread ID

* Upgrade reanimated to 2.3.0-beta.1 to fix JNI crash

* Remove `this_thread::get_id()`

* Update Podfile.lock
 2021-09-24 16:57:12 +02:00
Marc Rousavy
be3312e47d
chore: Upgrade kotlinx-coroutines 1.5.0 -> 1.5.2 (#450) 
* chore: Upgrade kotlinx-coroutines 1.5.0 -> 1.5.2

* Upgrade `kotlinVersion` to 1.5.30
 2021-09-23 14:22:11 +02:00
Marc Rousavy
e621422389 chore: Update Podfile.lock 2021-09-22 14:07:37 +02:00