61 Commits

Author SHA1 Message Date
Marc Rousavy
b24b1c808f docs: Fix links 2023-10-03 11:31:37 +02:00
Marc Rousavy
ce07750dea docs: Fix runAsync example 2023-09-30 12:12:43 +02:00
Marc Rousavy
2c5c7d63b1
docs: Don't use bold links (#1860) 2023-09-26 14:42:22 +02:00
Marc Rousavy
688963954a
docs: Align Images perfectly on mobile (#1856)
* docs: Use Light codeblocks theme

* docs: Fix image align

* fix pixels

* fix: Adjust all images
2023-09-26 13:09:44 +02:00
Marc Rousavy
14721d314f
chore: Remove semicolons (#1846)
* chore: Disable `semi` in Prettier

* chore: Format w/o semi

* Remove more `;`

* Lint example

* More ;
2023-09-26 11:39:17 +02:00
Marc Rousavy
2d66d5893c
docs: New V3 docs for new API (#1842)
* docs: New V3 docs for new API

* fix: Prefer Wide-Angle unless explicitly opted-out

* docs: Update DEVICES

* Finish Devices docs

* Switch links

* Revert "Switch links"

This reverts commit 06f196ae0e67787cbd5768e125be6d0a3cb5bbc9.

* docs: New LIFECYCLE

* docs: New CAPTURING docs

* Update Worklets links

* docs: Update TROUBLESHOOTING and ZOOMING

* fix: Update `getAvailableCameraDevices()` usages

* docs: Update FORMATS

* Update Errors.kt

* docs: Fix broken links

* docs: Update references to old hooks

* docs: Create Frame Processor Tips

* docs: Auto-dark mode

* fix: Fix FPS filter

* feat: Add `'max'` flag to format filter

* fix: Use loop

* fix: Fix bug in `getCameraFormat`

* fix: Find best aspect ratio as well

* fix: Switch between formats on FPS change

* Update FRAME_PROCESSOR_PLUGIN_LIST.mdx

* Add FPS graph explanation

* feat: Support HDR filter

* docs: Add HDR docs

* docs: Add Video Stabilization

* docs: Update Skia docs

* Skia links

* Add Skia labels

* Update SKIA_FRAME_PROCESSORS.mdx

* docs: Add Performance

* Update some wording

* Update headers / and zoom

* Add examples for devices

* fix highlights

* fix: Expose `Frame`

* docs: Update FP docs

* Update links

* Update FRAME_PROCESSOR_CREATE_PLUGIN_IOS.mdx
2023-09-25 12:57:03 +02:00
Marc Rousavy
036856aed5
chore: Move everything into package/ (#1745)
* Move everything into package

* Remove .DS_Store

* Move scripts and eslintrc to package

* Create CODE_OF_CONDUCT.md

* fix some links

* Update all links (I think)

* Update generated docs

* Update notice-yarn-changes.yml

* Update validate-android.yml

* Update validate-cpp.yml

* Delete notice-yarn-changes.yml

* Update validate-cpp.yml

* Update validate-cpp.yml

* Update validate-js.yml

* Update validate-cpp.yml

* Update validate-cpp.yml

* wrong c++ style

* Revert "wrong c++ style"

This reverts commit 55a3575589c6f13f8b05134d83384f55e0601ab2.
2023-09-01 18:15:28 +02:00
Marc Rousavy
2a5c33323b
docs: New README (#1744)
* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Revert "Update README.md"

This reverts commit 8ce4949388323dc8d81366972b350476b77a8b99.

* Update README.md

* Update README.md

* Update README.md

* Revert "Update README.md"

This reverts commit 11c4655890d78728cff3ab31b8578f86ba6cbf0f.

* Delete settings.json

* Delete CODE_OF_CONDUCT.md

* Move `.clang-format` to `cpp/`

* Update README.md

* update docs

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update sidebars.js
2023-09-01 17:31:10 +02:00
Marc Rousavy
862e05b64f
feat: Allow build without Skia or Frame Processors (#1710)
* feat: Make Frame Processors optional in JS

* Allow Android build without Frame Processors

* fix: Fix `EncoderProfiles.width` null-error

* Update gradle.properties

* Update gradle.properties

* fix: Use `#ifdef` instead of `#if`

* Update JVisionCameraProxy.cpp

* fix: Fix definitions

* Revert "fix: Use `#ifdef` instead of `#if`"

This reverts commit b19f32e5ce7df558cadcc8c4b5006c9cdf2cbe66.

* fix: Fix build

* chore: Codestyle

* Update JFrameProcessor.cpp
2023-08-23 12:42:38 +02:00
Marc Rousavy
37a3548a81
feat: Full Android rewrite (CameraX -> Camera2) (#1674)
* Nuke CameraX

* fix: Run View Finder on UI Thread

* Open Camera, set up Threads

* fix init

* Mirror if needed

* Try PreviewView

* Use max resolution

* Add `hardwareLevel` property

* Check if output type is supported

* Replace `frameRateRanges` with `minFps` and `maxFps`

* Remove `isHighestPhotoQualitySupported`

* Remove `colorSpace`

The native platforms will use the best / most accurate colorSpace by default anyways.

* HDR

* Check from format

* fix

* Remove `supportsParallelVideoProcessing`

* Correctly return video/photo sizes on Android now. Finally

* Log all Device props

* Log if optimized usecase is used

* Cleanup

* Configure Camera Input only once

* Revert "Configure Camera Input only once"

This reverts commit 0fd6c03f54c7566cb5592053720c4a8743aba92e.

* Extract Camera configuration

* Try to reconfigure all

* Hook based

* Properly set up `CameraSession`

* Delete unused

* fix: Fix recreate when outputs change

* Update NativePreviewView.kt

* Use callback for closing

* Catch CameraAccessException

* Finally got it stable

* Remove isMirrored

* Implement `takePhoto()`

* Add ExifInterface library

* Run findViewById on UI Thread

* Add Photo Output Surface to takePhoto

* Fix Video Stabilization Modes

* Optimize Imports

* More logs

* Update CameraSession.kt

* Close Image

* Use separate Executor in CameraQueue

* Delete hooks

* Use same Thread again

* If opened, call error

* Update CameraSession.kt

* Log HW level

* fix: Don't enable Stream Use Case if it's not 100% supported

* Move some stuff

* Cleanup PhotoOutputSynchronizer

* Try just open in suspend fun

* Some synchronization fixes

* fix logs

* Update CameraDevice+createCaptureSession.kt

* Update CameraDevice+createCaptureSession.kt

* fixes

* fix: Use Snapshot Template for speed capture prio

* Use PREVIEW template for repeating request

* Use `TEMPLATE_RECORD` if video use-case is attached

* Use `isRunning` flag

* Recreate session everytime on active/inactive

* Lazily get values in capture session

* Stability

* Rebuild session if outputs change

* Set `didOutputsChange` back to false

* Capture first in lock

* Try

* kinda fix it? idk

* fix: Keep Outputs

* Refactor into single method

* Update CameraView.kt

* Use Enums for type safety

* Implement Orientation (I think)

* Move RefCount management to Java (Frame)

* Don't crash when dropping a Frame

* Prefer Devices with higher max resolution

* Prefer multi-cams

* Use FastImage for Media Page

* Return orientation in takePhoto()

* Load orientation from EXIF Data

* Add `isMirrored` props and documentation for PhotoFile

* fix: Return `not-determined` on Android

* Update CameraViewModule.kt

* chore: Upgrade packages

* fix: Fix Metro Config

* Cleanup config

* Properly mirror Images on save

* Prepare MediaRecorder

* Start/Stop MediaRecorder

* Remove `takeSnapshot()`

It no longer works on Android and never worked on iOS. Users could use useFrameProcessor to take a Snapshot

* Use `MediaCodec`

* Move to `VideoRecording` class

* Cleanup Snapshot

* Create `SkiaPreviewView` hybrid class

* Create OpenGL context

* Create `SkiaPreviewView`

* Fix texture creation missing context

* Draw red frame

* Somehow get it working

* Add Skia CMake setup

* Start looping

* Init OpenGL

* Refactor into `SkiaRenderer`

* Cleanup PreviewSize

* Set up

* Only re-render UI if there is a new Frame

* Preview

* Fix init

* Try rendering Preview

* Update SkiaPreviewView.kt

* Log version

* Try using Skia (fail)

* Drawwwww!!!!!!!!!! 🎉

* Use Preview Size

* Clear first

* Refactor into SkiaRenderer

* Add `previewType: "none"` on iOS

* Simplify a lot

* Draw Camera? For some reason? I have no idea anymore

* Fix OpenGL errors

* Got it kinda working again?

* Actually draw Frame woah

* Clean up code

* Cleanup

* Update on main

* Synchronize render calls

* holy shit

* Update SkiaRenderer.cpp

* Update SkiaRenderer.cpp

* Refactor

* Update SkiaRenderer.cpp

* Check for `NO_INPUT_TEXTURE`^

* Post & Wait

* Set input size

* Add Video back again

* Allow session without preview

* Convert JPEG to byte[]

* feat: Use `ImageReader` and use YUV Image Buffers in Skia Context (#1689)

* Try to pass YUV Buffers as Pixmaps

* Create pixmap!

* Clean up

* Render to preview

* Only render if we have an output surface

* Update SkiaRenderer.cpp

* Fix Y+U+V sampling code

* Cleanup

* Fix Semaphore 0

* Use 4:2:0 YUV again idk

* Update SkiaRenderer.h

* Set minSdk to 26

* Set surface

* Revert "Set minSdk to 26"

This reverts commit c4085b7c16c628532e5c2d68cf7ed11c751d0b48.

* Set previewType

* feat: Video Recording with Camera2 (#1691)

* Rename

* Update CameraSession.kt

* Use `SurfaceHolder` instead of `SurfaceView` for output

* Update CameraOutputs.kt

* Update CameraSession.kt

* fix: Fix crash when Preview is null

* Check if snapshot capture is supported

* Update RecordingSession.kt

* S

* Use `MediaRecorder`

* Make audio optional

* Add Torch

* Output duration

* Update RecordingSession.kt

* Start RecordingSession

* logs

* More log

* Base for preparing pass-through Recording

* Use `ImageWriter` to append Images to the Recording Surface

* Stream PRIVATE GPU_SAMPLED_IMAGE Images

* Add flags

* Close session on stop

* Allow customizing `videoCodec` and `fileType`

* Enable Torch

* Fix Torch Mode

* Fix comparing outputs with hashCode

* Update CameraSession.kt

* Correctly pass along Frame Processor

* fix: Use AUDIO_BIT_RATE of 16 * 44,1Khz

* Use CAMCORDER instead of MIC microphone

* Use 1 channel

* fix: Use `Orientation`

* Add `native` PixelFormat

* Update iOS to latest Skia integration

* feat: Add `pixelFormat` property to Camera

* Catch error in configureSession

* Fix JPEG format

* Clean up best match finder

* Update CameraDeviceDetails.kt

* Clamp sizes by maximum CamcorderProfile size

* Remove `getAvailableVideoCodecs`

* chore: release 3.0.0-rc.5

* Use maximum video size of RECORD as default

* Update CameraDeviceDetails.kt

* Add a todo

* Add JSON device to issue report

* Prefer `full` devices and flash

* Lock to 30 FPS on Samsung

* Implement Zoom

* Refactor

* Format -> PixelFormat

* fix: Feat `pixelFormat` -> `pixelFormats`

* Update TROUBLESHOOTING.mdx

* Format

* fix: Implement `zoom` for Photo Capture

* fix: Don't run if `isActive` is `false`

* fix: Call `examplePlugin(frame)`

* fix: Fix Flash

* fix: Use `react-native-worklets-core`!

* fix: Fix import
2023-08-21 12:50:14 +02:00
Marc Rousavy
44ed42d5d6
feat: Expose unified VisionCameraProxy object, make FrameProcessorPlugins object-oriented (#1660)
* feat: Replace `FrameProcessorRuntimeManager` with `VisionCameraProxy` (iOS)

* Make `FrameProcessorPlugin` a constructable HostObject

* fix: Fix `name` override

* Simplify `useFrameProcessor

* fix: Fix lint errors

* Remove FrameProcessorPlugin::name

* JSIUtils -> JSINSObjectConversion
2023-07-21 17:52:30 +02:00
Marc Rousavy
30b56153db
feat: Sync Frame Processors (plus runAsync and runAtTargetFps) (#1472)
Before, Frame Processors ran on a separate Thread.

After, Frame Processors run fully synchronous and always at the same FPS as the Camera.

Two new functions have been introduced:

* `runAtTargetFps(fps: number, func: () => void)`: Runs the given code as often as the given `fps`, effectively throttling it's calls.
* `runAsync(frame: Frame, func: () => void)`: Runs the given function on a separate Thread for Frame Processing. A strong reference to the Frame is held as long as the function takes to execute.

You can use `runAtTargetFps` to throttle calls to a specific API (e.g. if your Camera is running at 60 FPS, but you only want to run face detection at ~25 FPS, use `runAtTargetFps(25, ...)`.)

You can use `runAsync` to run a heavy algorithm asynchronous, so that the Camera is not blocked while your algorithm runs. This is useful if your main sync processor draws something, and your async processor is doing some image analysis on the side. 

You can also combine both functions.

Examples:

```js
const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  console.log("I'm running at 60 FPS!")
}, [])
```

```js
const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  console.log("I'm running at 60 FPS!")

  runAtTargetFps(10, () => {
    'worklet'
    console.log("I'm running at 10 FPS!")
  })
}, [])
```



```js
const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  console.log("I'm running at 60 FPS!")

  runAsync(frame, () => {
    'worklet'
    console.log("I'm running on another Thread, I can block for longer!")
  })
}, [])
```

```js
const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  console.log("I'm running at 60 FPS!")

  runAtTargetFps(10, () => {
    'worklet'
    runAsync(frame, () => {
      'worklet'
      console.log("I'm running on another Thread at 10 FPS, I can block for longer!")
    })
  })
}, [])
```
2023-02-15 16:47:09 +01:00
Marc Rousavy
a0590dccb5
feat: Replace Reanimated with RN Worklets (#1468)
* Setup RN Worklets

* Use RN Worklets on iOS

* Fix console

* Add `installFrameProcessorBindings()` function

* Add `FrameProcessorPlugins` proxy (BREAKING CHANGE)

* Clean up docs

* Update FRAME_PROCESSORS.mdx

* Use RN Worklets 0.2.5

* feat: Android build setup

* Rewrite Android Frame Processor Part

* Update CMakeLists.txt

* fix: Add react-native-worklets Gradle dependency

* Update Podfile.lock

* fix build

* gradle:7.4.1

* Init JSI Bindings in method on Android

* Fix Folly flags

* fix: Init `FrameProcessorRuntimeManager` later

* fix: Wrap in `<GestureHandlerRootView>`

* Refactor plugins

* fix: Remove enableFrameProcessors

* Install RN Worklets from current GH master

* Update babel.config.js

* Update CameraViewModule.kt

* Update ImageProxyUtils.java

* feat: Upgrade to Reanimated v3

* fix: Fix crash on Worklet init

* Update RN Worklets to latest master

* fix: Simplify FP Plugins Proxy
2023-02-13 15:22:45 +01:00
Rohan Barman
6b32cd9cd7
docs: Replace outdated react-native-reanimated doc links (#1304) 2022-10-28 12:18:36 +02:00
Marc Rousavy
13516c706b Update FRAME_PROCESSORS.mdx 2022-01-11 12:41:41 +01:00
Marc Rousavy
a0b21775e3 Update FRAME_PROCESSORS.mdx 2022-01-11 12:41:19 +01:00
Marc Rousavy
d44ed2ea39 Update FRAME_PROCESSORS.mdx 2022-01-11 12:39:05 +01:00
Marc Rousavy
c2518bc40f docs: Add documentation for disabling Frame Processors on Expo 2022-01-11 12:38:32 +01:00
Marc Rousavy
be5ec69b02
feat: Make Reanimated optional (disable Frame Processors if REA v2 is not installed) (#412)
* Fix building iOS without Reanimated

* Conditionally compile Frame Processors (gradle)

* Conditionally use externalNativeBuild

* Remove Reanimated import

* fix: Conditionally load REA/VisionCamera libraries

* fix: Add disable FP to docs

* fix: Fix dummy placeholder for Scheduler.mm

* fix: Fix dummy `Scheduler` declaration

* fix: Only init `CameraView` C++ side if frame processors are enabled

* fix: Install JSI Bindings on Frame Processor Manager ctor

* fix: Wrong conditional

* whoops
2022-01-02 17:35:26 +01:00
Marc Rousavy
934de142ee docs: Update Format.pixelFormat documentation 2021-12-10 09:52:40 +01:00
Marc Rousavy
d3105fa207
chore: Upgrade a whole lotta dependencies (#436)
* chore: Upgrade a lot of dependencies for `./`

* chore: Upgrade a lot of dependencies for `./example`

* chore: Upgrade a lot of dependencies for `./docs`

* Use new `EventEmitter` syntax (`.remove()`)

* Update Podfile.lock

* docs: Use watch mode

* docs: Replace all relative links with absolute

* Fix all links

* Update docusaurus.config.js

* Upgrade docusaurus-plugin-typedoc to fix docs build

* Update yarn.lock

* Upgrade typescript to 4.4.3

* Fix error unknown

* Update package.json

* Upgrade typedoc

* Upgrade a few more deps

* Fix deprecated sidebar syntax

* Update Gemfile.lock
2021-09-22 13:58:59 +02:00
Marc Rousavy
68d737d6df
Update FRAME_PROCESSORS.mdx 2021-09-13 10:40:46 +02:00
Marc Rousavy
3aa1f5f849
Update FRAME_PROCESSORS.mdx 2021-09-09 22:04:33 +02:00
Marc Rousavy
b5452ac406
Update FRAME_PROCESSORS.mdx 2021-08-10 14:10:07 +02:00
Marc Rousavy
cc4d9c519b
docs: Improve cat bounds example 2021-08-10 14:08:52 +02:00
Marc Rousavy
cbcde138fb
Update FRAME_PROCESSORS.mdx 2021-07-26 15:41:03 +02:00
Marc Rousavy
25ac233088
Update FRAME_PROCESSORS.mdx 2021-07-09 10:21:02 +02:00
Marc Rousavy
9ef2496a7a
Update FRAME_PROCESSORS.mdx 2021-07-08 14:23:30 +02:00
Marc Rousavy
a62697fe2a
Update FRAME_PROCESSORS.mdx 2021-07-08 14:22:56 +02:00
Marc Rousavy
cf68a4c647
docs: Explain Frame Processor benefits more 2021-07-08 14:17:10 +02:00
Marc Rousavy
ba50e2b485
Update FRAME_PROCESSORS.mdx 2021-07-07 10:21:17 +02:00
Marc Rousavy
1f680276a0
Update FRAME_PROCESSORS.mdx 2021-07-06 15:16:54 +02:00
Marc Rousavy
3e3485ff7e
Update FRAME_PROCESSORS.mdx 2021-07-06 15:15:57 +02:00
Marc Rousavy
18e0d3ec7d
docs: Re-order Frame Processor docs 2021-07-06 15:12:29 +02:00
Marc Rousavy
db9c7d8e00
Update FRAME_PROCESSORS.mdx 2021-07-06 15:06:14 +02:00
Marc Rousavy
4a362a544f
Update FRAME_PROCESSORS.mdx 2021-07-06 15:05:13 +02:00
Marc Rousavy
d4057e9d8e
Update FRAME_PROCESSORS.mdx 2021-07-06 15:03:40 +02:00
Marc Rousavy
5ad386bfbc
Update FRAME_PROCESSORS.mdx 2021-07-06 15:01:58 +02:00
Marc Rousavy
94914907ba
docs: Shorten code sample 2021-07-06 14:56:44 +02:00
Marc Rousavy
1646bb56f4
docs: Link to Worklets docs 2021-07-06 14:54:05 +02:00
Marc Rousavy
db0baaefe8
docs: Use Hotdog/Not Hotdog as sample 2021-07-06 14:52:28 +02:00
Marc Rousavy
b39b38006c
docs: Update "The Frame object" docs 2021-07-06 14:32:39 +02:00
Marc Rousavy
0f6d0978f0
docs: Update Frame Processor Technical docs 2021-07-06 10:37:56 +02:00
Marc Rousavy
2a00f2a098
Update FRAME_PROCESSORS.mdx 2021-06-24 12:07:15 +02:00
Marc Rousavy
7164b34510
Update FRAME_PROCESSORS.mdx 2021-06-24 12:05:52 +02:00
Marc Rousavy
48a97eccdd docs: Move ANIMATED -> ZOOMING 2021-06-09 12:46:26 +02:00
Marc Rousavy
5508dae8b6
docs: Add missing dependency 2021-06-08 14:06:26 +02:00
Marc Rousavy
a02f378a4b
docs: Lifecycle (#182)
* Add docs for Lifecycle

* Update CAPTURING.mdx

* move

* Update DEVICES.mdx

* Update FRAME_PROCESSOR_PLUGIN_LIST.mdx

* f

* move FP

* separate focusing

* fix links
2021-06-07 15:55:20 +02:00
Marc Rousavy
8189173120
Update FRAME_PROCESSORS.mdx 2021-05-31 14:11:33 +02:00
Marc Rousavy
4118fd17eb
docs: Add Frame Processor benchmarks (#154)
* remove snapcode docs

* Add benchmarks

* Update FRAME_PROCESSORS.mdx
2021-05-30 12:13:50 +02:00