Commit Graph

28 Commits

Author SHA1 Message Date
Marc Rousavy
37a3548a81
feat: Full Android rewrite (CameraX -> Camera2) (#1674)
* Nuke CameraX

* fix: Run View Finder on UI Thread

* Open Camera, set up Threads

* fix init

* Mirror if needed

* Try PreviewView

* Use max resolution

* Add `hardwareLevel` property

* Check if output type is supported

* Replace `frameRateRanges` with `minFps` and `maxFps`

* Remove `isHighestPhotoQualitySupported`

* Remove `colorSpace`

The native platforms will use the best / most accurate colorSpace by default anyways.

* HDR

* Check from format

* fix

* Remove `supportsParallelVideoProcessing`

* Correctly return video/photo sizes on Android now. Finally

* Log all Device props

* Log if optimized usecase is used

* Cleanup

* Configure Camera Input only once

* Revert "Configure Camera Input only once"

This reverts commit 0fd6c03f54c7566cb5592053720c4a8743aba92e.

* Extract Camera configuration

* Try to reconfigure all

* Hook based

* Properly set up `CameraSession`

* Delete unused

* fix: Fix recreate when outputs change

* Update NativePreviewView.kt

* Use callback for closing

* Catch CameraAccessException

* Finally got it stable

* Remove isMirrored

* Implement `takePhoto()`

* Add ExifInterface library

* Run findViewById on UI Thread

* Add Photo Output Surface to takePhoto

* Fix Video Stabilization Modes

* Optimize Imports

* More logs

* Update CameraSession.kt

* Close Image

* Use separate Executor in CameraQueue

* Delete hooks

* Use same Thread again

* If opened, call error

* Update CameraSession.kt

* Log HW level

* fix: Don't enable Stream Use Case if it's not 100% supported

* Move some stuff

* Cleanup PhotoOutputSynchronizer

* Try just open in suspend fun

* Some synchronization fixes

* fix logs

* Update CameraDevice+createCaptureSession.kt

* Update CameraDevice+createCaptureSession.kt

* fixes

* fix: Use Snapshot Template for speed capture prio

* Use PREVIEW template for repeating request

* Use `TEMPLATE_RECORD` if video use-case is attached

* Use `isRunning` flag

* Recreate session everytime on active/inactive

* Lazily get values in capture session

* Stability

* Rebuild session if outputs change

* Set `didOutputsChange` back to false

* Capture first in lock

* Try

* kinda fix it? idk

* fix: Keep Outputs

* Refactor into single method

* Update CameraView.kt

* Use Enums for type safety

* Implement Orientation (I think)

* Move RefCount management to Java (Frame)

* Don't crash when dropping a Frame

* Prefer Devices with higher max resolution

* Prefer multi-cams

* Use FastImage for Media Page

* Return orientation in takePhoto()

* Load orientation from EXIF Data

* Add `isMirrored` props and documentation for PhotoFile

* fix: Return `not-determined` on Android

* Update CameraViewModule.kt

* chore: Upgrade packages

* fix: Fix Metro Config

* Cleanup config

* Properly mirror Images on save

* Prepare MediaRecorder

* Start/Stop MediaRecorder

* Remove `takeSnapshot()`

It no longer works on Android and never worked on iOS. Users could use useFrameProcessor to take a Snapshot

* Use `MediaCodec`

* Move to `VideoRecording` class

* Cleanup Snapshot

* Create `SkiaPreviewView` hybrid class

* Create OpenGL context

* Create `SkiaPreviewView`

* Fix texture creation missing context

* Draw red frame

* Somehow get it working

* Add Skia CMake setup

* Start looping

* Init OpenGL

* Refactor into `SkiaRenderer`

* Cleanup PreviewSize

* Set up

* Only re-render UI if there is a new Frame

* Preview

* Fix init

* Try rendering Preview

* Update SkiaPreviewView.kt

* Log version

* Try using Skia (fail)

* Drawwwww!!!!!!!!!! 🎉

* Use Preview Size

* Clear first

* Refactor into SkiaRenderer

* Add `previewType: "none"` on iOS

* Simplify a lot

* Draw Camera? For some reason? I have no idea anymore

* Fix OpenGL errors

* Got it kinda working again?

* Actually draw Frame woah

* Clean up code

* Cleanup

* Update on main

* Synchronize render calls

* holy shit

* Update SkiaRenderer.cpp

* Update SkiaRenderer.cpp

* Refactor

* Update SkiaRenderer.cpp

* Check for `NO_INPUT_TEXTURE`^

* Post & Wait

* Set input size

* Add Video back again

* Allow session without preview

* Convert JPEG to byte[]

* feat: Use `ImageReader` and use YUV Image Buffers in Skia Context (#1689)

* Try to pass YUV Buffers as Pixmaps

* Create pixmap!

* Clean up

* Render to preview

* Only render if we have an output surface

* Update SkiaRenderer.cpp

* Fix Y+U+V sampling code

* Cleanup

* Fix Semaphore 0

* Use 4:2:0 YUV again idk

* Update SkiaRenderer.h

* Set minSdk to 26

* Set surface

* Revert "Set minSdk to 26"

This reverts commit c4085b7c16c628532e5c2d68cf7ed11c751d0b48.

* Set previewType

* feat: Video Recording with Camera2 (#1691)

* Rename

* Update CameraSession.kt

* Use `SurfaceHolder` instead of `SurfaceView` for output

* Update CameraOutputs.kt

* Update CameraSession.kt

* fix: Fix crash when Preview is null

* Check if snapshot capture is supported

* Update RecordingSession.kt

* S

* Use `MediaRecorder`

* Make audio optional

* Add Torch

* Output duration

* Update RecordingSession.kt

* Start RecordingSession

* logs

* More log

* Base for preparing pass-through Recording

* Use `ImageWriter` to append Images to the Recording Surface

* Stream PRIVATE GPU_SAMPLED_IMAGE Images

* Add flags

* Close session on stop

* Allow customizing `videoCodec` and `fileType`

* Enable Torch

* Fix Torch Mode

* Fix comparing outputs with hashCode

* Update CameraSession.kt

* Correctly pass along Frame Processor

* fix: Use AUDIO_BIT_RATE of 16 * 44,1Khz

* Use CAMCORDER instead of MIC microphone

* Use 1 channel

* fix: Use `Orientation`

* Add `native` PixelFormat

* Update iOS to latest Skia integration

* feat: Add `pixelFormat` property to Camera

* Catch error in configureSession

* Fix JPEG format

* Clean up best match finder

* Update CameraDeviceDetails.kt

* Clamp sizes by maximum CamcorderProfile size

* Remove `getAvailableVideoCodecs`

* chore: release 3.0.0-rc.5

* Use maximum video size of RECORD as default

* Update CameraDeviceDetails.kt

* Add a todo

* Add JSON device to issue report

* Prefer `full` devices and flash

* Lock to 30 FPS on Samsung

* Implement Zoom

* Refactor

* Format -> PixelFormat

* fix: Feat `pixelFormat` -> `pixelFormats`

* Update TROUBLESHOOTING.mdx

* Format

* fix: Implement `zoom` for Photo Capture

* fix: Don't run if `isActive` is `false`

* fix: Call `examplePlugin(frame)`

* fix: Fix Flash

* fix: Use `react-native-worklets-core`!

* fix: Fix import
2023-08-21 12:50:14 +02:00
Marc Rousavy
757ecb0640 docs: Deploy to https://react-native-vision-camera.com 2023-03-06 11:03:40 +01:00
Marc Rousavy
934de142ee docs: Update Format.pixelFormat documentation 2021-12-10 09:52:40 +01:00
Thomas Coldwell
c3039c94c6
feat: Add pixelFormat field (#559)
* Add `mediaSubType` field to device formats

* Create FourCharCode `toString` extension

* Move pixelFormat -> `AVCaptureDevice.Format+toDictionary`

* `mediaSubType` -> `pixelFormat` + non-optional

* ts pixelFormat `string` -> specific formats

* Add default pixelFormat value of `420v` on Android

Co-authored-by: tcoldwell72 <31568400+tcoldwell72@users.noreply.github.com>
2021-12-10 09:44:30 +01:00
Marc Rousavy
98f1961fa7
fix: Throw 'frame-processor/unavailable' error instead of normal JS Error (#461)
* Throw `'frame-processor/unavailable'` error instead of normal JS Error

* Update CameraDevice.ts
2021-09-27 14:44:09 +02:00
Marc Rousavy
445af943c3
feat: BREAKING CHANGE: Express zoom factor always in actual factor value (1, 2, 128, ...) instead of 0.0-1.0 scale (#306)
* Make `zoom` go on "factor" scale

* Clean up `zoom` code

* fix float conversion

* fix zoom interpretation

* Update docs for new zoom scale

* fix float conversion
2021-07-29 11:44:22 +02:00
Marc Rousavy
b8b1a5c547 docs: Improve CameraDevices documentation 2021-07-29 10:14:50 +02:00
Marc Rousavy
5126dd63b4 feat: Make videoWidth and videoHeight available on all devices 2021-07-08 10:59:27 +02:00
Marc Rousavy
87e6bb710e
feat: Frame Processors for Android (#196)
* Create android gradle build setup

* Fix `prefab` config

* Add `pickFirst **/*.so` to example build.gradle

* fix REA path

* cache gradle builds

* Update validate-android.yml

* Create Native Proxy

* Copy REA header

* implement ctor

* Rename CameraViewModule -> FrameProcessorRuntimeManager

* init FrameProcessorRuntimeManager

* fix name

* Update FrameProcessorRuntimeManager.h

* format

* Create AndroidErrorHandler.h

* Initialize runtime and install JSI funcs

* Update FrameProcessorRuntimeManager.cpp

* Update CameraViewModule.kt

* Make CameraView hybrid C++ class to find view & set frame processor

* Update FrameProcessorRuntimeManager.cpp

* pass function by rvalue

* pass by const &&

* extract hermes and JSC REA

* pass `FOR_HERMES`

* correctly prepare JSC and Hermes

* Update CMakeLists.txt

* add missing hermes include

* clean up imports

* Create JImageProxy.h

* pass ImageProxy to JNI as `jobject`

* try use `JImageProxy` C++ wrapper type

* Use `local_ref<JImageProxy>`

* Create `JImageProxyHostObject` for JSI interop

* debug call to frame processor

* Unset frame processor

* Fix CameraView native part not being registered

* close image

* use `jobject` instead of `JImageProxy` for now :(

* fix hermes build error

* Set enable FP callback

* fix JNI call

* Update CameraView.cpp

* Get Format

* Create plugin abstract

* Make `FrameProcessorPlugin` a hybrid object

* Register plugin CXX

* Call `registerPlugin`

* Catch

* remove JSI

* Create sample QR code plugin

* register plugins

* Fix missing JNI binding

* Add `mHybridData`

* prefix name with two underscores (`__`)

* Update CameraPage.tsx

* wrap `ImageProxy` in host object

* Use `jobject` for HO box

* Update JImageProxy.h

* reinterpret jobject

* Try using `JImageProxy` instead of `jobject`

* Update JImageProxy.h

* get bytes per row and plane count

* Update CameraView.cpp

* Return base

* add some docs and JNI JSI conversion

* indent

* Convert JSI value to JNI jobject

* using namespace facebook

* Try using class

* Use plain old Object[]

* Try convert JNI -> JSI

* fix decl

* fix bool init

* Correctly link folly

* Update CMakeLists.txt

* Convert Map to Object

* Use folly for Map and Array

* Return `alias_ref<jobject>` instead of raw `jobject`

* fix JNI <-> JSI conversion

* Update JSIJNIConversion.cpp

* Log parameters

* fix params index offset

* add more test cases

* Update FRAME_PROCESSORS_CREATE_OVERVIEW.mdx

* fix types

* Rename to example plugin

* remove support for hashmap

* Try use HashMap iterable fbjni binding

* try using JReadableArray/JReadableMap

* Fix list return values

* Update JSIJNIConversion.cpp

* Update JSIJNIConversion.cpp

* (iOS) Rename ObjC QR Code Plugin to Example Plugin

* Rename Swift plugin QR -> Example

* Update ExamplePluginSwift.swift

* Fix Map/Dictionary logging format

* Update ExampleFrameProcessorPlugin.m

* Reconfigure session if frame processor changed

* Handle use-cases via `maxUseCasesCount`

* Don't crash app on `configureSession` error

* Document "use-cases"

* Update DEVICES.mdx

* fix merge

* Make `const &`

* iOS: Automatically enable `video` if a `frameProcessor` is set

* Update CameraView.cpp

* fix docs

* Automatically fallback to snapshot capture if `supportsParallelVideoProcessing` is false.

* Fix lookup

* Update CameraView.kt

* Implement `frameProcessorFps`

* Finalize Frame Processor Plugin Hybrid

* Update CameraViewModule.kt

* Support `flash` on `takeSnapshot()`

* Update docs

* Add docs

* Update CameraPage.tsx

* Attribute NonNull

* remove unused imports

* Add Android docs for Frame Processors

* Make JNI HashMap <-> JSI Object conversion faster

directly access `toHashMap` instead of going through java

* add todo

* Always run `prepareJSC` and `prepareHermes`

* switch jsc and hermes

* Specify ndkVersion `21.4.7075529`

* Update gradle.properties

* Update gradle.properties

* Create .aar

* Correctly prepare android package

* Update package.json

* Update package.json

* remove `prefab` build feature

* split

* Add docs for registering the FP plugin

* Add step for dep

* Update CaptureButton.tsx

* Move to `reanimated-headers/`

* Exclude reanimated-headers from cpplint

* disable `build/include_order` rule

* cpplint fixes

* perf: Make `JSIJNIConversion` a `namespace` instead of `class`

* Ignore runtime/references for `convert` funcs

* Build Android .aar in CI

* Run android build script only on `prepack`

* Update package.json

* Update package.json

* Update build-android-npm-package.sh

* Move to `yarn build`

* Also install node_modules in example step

* Update validate-android.yml

* sort imports

* fix torch

* Run ImageAnalysis on `FrameProcessorThread`

* Update Errors.kt

* Add clean android script

* Upgrade reanimated to 2.3.0-alpha.1

* Revert "Upgrade reanimated to 2.3.0-alpha.1"

This reverts commit c1d3bed5e03728d0b5e335a359524ff4f56f5035.

* ⚠️ TEMP FIX: hotfix reanimated build.gradle

* Update CameraView+TakeSnapshot.kt

* ⚠️ TEMP FIX: Disable ktlint action for now

* Update clean.sh

* Set max heap size to 4g

* rebuild lockfiles

* Update Podfile.lock

* rename

* Build lib .aar before example/
2021-06-27 12:37:54 +02:00
Marc Rousavy
9ea158ad8f
chore: Move to /mrousavy/ (#224)
* rename 1/n

* 2

* 3

* fix indent
2021-06-21 22:42:46 +02:00
Marc Rousavy
a02f378a4b
docs: Lifecycle (#182)
* Add docs for Lifecycle

* Update CAPTURING.mdx

* move

* Update DEVICES.mdx

* Update FRAME_PROCESSOR_PLUGIN_LIST.mdx

* f

* move FP

* separate focusing

* fix links
2021-06-07 15:55:20 +02:00
Marc Rousavy
45e7105bdf
docs: Add api link (#181)
* Add API link to `supportsPhotoAndVideoCapture`

* Update DEVICES.mdx

* and -> or

* add link

* Update CAPTURING.mdx

* Update DEVICES.mdx

* Make links relative
2021-06-07 13:39:36 +02:00
Marc Rousavy
72a1fad78e
feat: Separate usecases (decouple microphone, video, photo) (#168)
* Add props

* add props (iOS)

* Add use-cases conditionally

* Update CameraView+RecordVideo.swift

* Update RecordingSession.swift

* reconfigure on change

* Throw correct errors

* Check for audio permission

* Move `#if` outward

* Throw appropriate errors

* Update CameraView+RecordVideo.swift

* fix Splashscreen

* Dynamic filePath

* Fix video extension

* add `avci` and `m4v` file types

* Fix RecordVideo errors

* Fix audio setup

* Enable `photo`, `video` and `audio`

* Check for `video={true}` in frameProcessor

* format

* Remove unused DispatchQueue

* Update docs

* Add `supportsPhotoAndVideoCapture`

* Fix view manager

* Fix error not being propagated

* Catch normal errors too

* Update DEVICES.mdx

* Update CAPTURING.mdx

* Update classdocs
2021-06-07 13:08:40 +02:00
Marc Rousavy
555474be7d
fix: Represent neutralZoom in factor instead of percentage (#179)
* Use factor instead of percent for `neutralZoom`

* fix zoom calculation

* Update CameraPage.tsx
2021-06-07 10:46:53 +02:00
Marc Rousavy
9320e356e3
Get sizes per ImageFormat/PixelFormat (#119)
* Get sizes per ImageFormat/PixelFormat

* Fix `getOutputMinFrameDuration` crashing

* Fix `isHighestPhotoQualitySupported`

* Add TS type explanations

* Update CameraDevice.ts

* Update CameraViewModule.kt

* Add link to AVCaptureColorSpace apple docs
2021-04-13 13:01:24 +02:00
Marc Rousavy
7c33839e36
Reformat (#116)
* Reformat

* Prettier: Print Width 140

* SwiftFormat: Disable `organizeDeclarations`

* React Helpers -> React Utils

* Use bundle exec for pods

* Disable Hermes in Example

* Create clean.sh

* Update package.json

* Re-generate lockfiles

* Add bundle install step to bootstrap
2021-03-31 15:43:29 +02:00
Marc Rousavy
f0828e63f0
Fix docs broken links (#97)
* Fix broken CameraProps.format link

* Use absolute urls for SETUP.mdx
2021-03-23 15:49:58 +01:00
Marc Rousavy
48821d50ca
Move Guides to docs/guides directory (#96)
* Move Guides to docs/guides directory

* Rename sidebar

* Fix api/ links

* Update SETUP.mdx
2021-03-23 15:25:27 +01:00
Marc Rousavy
db8960580f
Feature/supports focus (#72)
* Add `supportsFocus` prop to CameraDevice

* Add true-depth camera again

* Revert "Add true-depth camera again"

This reverts commit 5814b8ca7a15b3273ff29498144bb352619a4c1d.
2021-03-17 18:07:05 +01:00
Marc Rousavy
9f30348053 Filter out true-depth-camera in getAvailableCameraDevices 2021-03-17 15:30:17 +01:00
Marc Rousavy
bfba6a95ee Use @linkcode for typedocs 2021-03-08 18:51:53 +01:00
Marc Rousavy
66aae62d17 Add more TypeDoc links 2021-03-08 18:21:30 +01:00
Marc Rousavy
f051481010
Devops/reviewdog typescript (#25)
* Use reviewdog for TSC

* Replace all `type Readonly<T>` with `interface T`

* Use --frozen-lockfile for yarn

* Give all actions a name

* Fix yarn example
2021-03-08 17:51:47 +01:00
Marc Rousavy
0b7b4d50b5
Automatically build API documentation (#11)
* Automatically build API documentation using Typedoc and Docusaurus
* Move MD and move to MDX for Docusaurus Guides
2021-03-03 12:37:43 +01:00
Marc Rousavy
4d8baf8cfa Update CameraDevice.ts 2021-02-20 17:11:17 +01:00
Marc Rousavy
dc2be934f6 ESLint autofix 2021-02-20 17:07:10 +01:00
Marc Rousavy
b2594f3e12 import type 2021-02-19 16:24:28 +01:00
Marc Rousavy
c04a4b72be Bootstrap 2021-02-19 16:07:53 +01:00