railbird/react-native-vision-camera
10
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat: Full Android rewrite (CameraX -> Camera2) (#1674)

Browse Source 
* Nuke CameraX

* fix: Run View Finder on UI Thread

* Open Camera, set up Threads

* fix init

* Mirror if needed

* Try PreviewView

* Use max resolution

* Add `hardwareLevel` property

* Check if output type is supported

* Replace `frameRateRanges` with `minFps` and `maxFps`

* Remove `isHighestPhotoQualitySupported`

* Remove `colorSpace`

The native platforms will use the best / most accurate colorSpace by default anyways.

* HDR

* Check from format

* fix

* Remove `supportsParallelVideoProcessing`

* Correctly return video/photo sizes on Android now. Finally

* Log all Device props

* Log if optimized usecase is used

* Cleanup

* Configure Camera Input only once

* Revert "Configure Camera Input only once"

This reverts commit 0fd6c03f54c7566cb5592053720c4a8743aba92e.

* Extract Camera configuration

* Try to reconfigure all

* Hook based

* Properly set up `CameraSession`

* Delete unused

* fix: Fix recreate when outputs change

* Update NativePreviewView.kt

* Use callback for closing

* Catch CameraAccessException

* Finally got it stable

* Remove isMirrored

* Implement `takePhoto()`

* Add ExifInterface library

* Run findViewById on UI Thread

* Add Photo Output Surface to takePhoto

* Fix Video Stabilization Modes

* Optimize Imports

* More logs

* Update CameraSession.kt

* Close Image

* Use separate Executor in CameraQueue

* Delete hooks

* Use same Thread again

* If opened, call error

* Update CameraSession.kt

* Log HW level

* fix: Don't enable Stream Use Case if it's not 100% supported

* Move some stuff

* Cleanup PhotoOutputSynchronizer

* Try just open in suspend fun

* Some synchronization fixes

* fix logs

* Update CameraDevice+createCaptureSession.kt

* Update CameraDevice+createCaptureSession.kt

* fixes

* fix: Use Snapshot Template for speed capture prio

* Use PREVIEW template for repeating request

* Use `TEMPLATE_RECORD` if video use-case is attached

* Use `isRunning` flag

* Recreate session everytime on active/inactive

* Lazily get values in capture session

* Stability

* Rebuild session if outputs change

* Set `didOutputsChange` back to false

* Capture first in lock

* Try

* kinda fix it? idk

* fix: Keep Outputs

* Refactor into single method

* Update CameraView.kt

* Use Enums for type safety

* Implement Orientation (I think)

* Move RefCount management to Java (Frame)

* Don't crash when dropping a Frame

* Prefer Devices with higher max resolution

* Prefer multi-cams

* Use FastImage for Media Page

* Return orientation in takePhoto()

* Load orientation from EXIF Data

* Add `isMirrored` props and documentation for PhotoFile

* fix: Return `not-determined` on Android

* Update CameraViewModule.kt

* chore: Upgrade packages

* fix: Fix Metro Config

* Cleanup config

* Properly mirror Images on save

* Prepare MediaRecorder

* Start/Stop MediaRecorder

* Remove `takeSnapshot()`

It no longer works on Android and never worked on iOS. Users could use useFrameProcessor to take a Snapshot

* Use `MediaCodec`

* Move to `VideoRecording` class

* Cleanup Snapshot

* Create `SkiaPreviewView` hybrid class

* Create OpenGL context

* Create `SkiaPreviewView`

* Fix texture creation missing context

* Draw red frame

* Somehow get it working

* Add Skia CMake setup

* Start looping

* Init OpenGL

* Refactor into `SkiaRenderer`

* Cleanup PreviewSize

* Set up

* Only re-render UI if there is a new Frame

* Preview

* Fix init

* Try rendering Preview

* Update SkiaPreviewView.kt

* Log version

* Try using Skia (fail)

* Drawwwww!!!!!!!!!! 🎉

* Use Preview Size

* Clear first

* Refactor into SkiaRenderer

* Add `previewType: "none"` on iOS

* Simplify a lot

* Draw Camera? For some reason? I have no idea anymore

* Fix OpenGL errors

* Got it kinda working again?

* Actually draw Frame woah

* Clean up code

* Cleanup

* Update on main

* Synchronize render calls

* holy shit

* Update SkiaRenderer.cpp

* Update SkiaRenderer.cpp

* Refactor

* Update SkiaRenderer.cpp

* Check for `NO_INPUT_TEXTURE`^

* Post & Wait

* Set input size

* Add Video back again

* Allow session without preview

* Convert JPEG to byte[]

* feat: Use `ImageReader` and use YUV Image Buffers in Skia Context (#1689)

* Try to pass YUV Buffers as Pixmaps

* Create pixmap!

* Clean up

* Render to preview

* Only render if we have an output surface

* Update SkiaRenderer.cpp

* Fix Y+U+V sampling code

* Cleanup

* Fix Semaphore 0

* Use 4:2:0 YUV again idk

* Update SkiaRenderer.h

* Set minSdk to 26

* Set surface

* Revert "Set minSdk to 26"

This reverts commit c4085b7c16c628532e5c2d68cf7ed11c751d0b48.

* Set previewType

* feat: Video Recording with Camera2 (#1691)

* Rename

* Update CameraSession.kt

* Use `SurfaceHolder` instead of `SurfaceView` for output

* Update CameraOutputs.kt

* Update CameraSession.kt

* fix: Fix crash when Preview is null

* Check if snapshot capture is supported

* Update RecordingSession.kt

* S

* Use `MediaRecorder`

* Make audio optional

* Add Torch

* Output duration

* Update RecordingSession.kt

* Start RecordingSession

* logs

* More log

* Base for preparing pass-through Recording

* Use `ImageWriter` to append Images to the Recording Surface

* Stream PRIVATE GPU_SAMPLED_IMAGE Images

* Add flags

* Close session on stop

* Allow customizing `videoCodec` and `fileType`

* Enable Torch

* Fix Torch Mode

* Fix comparing outputs with hashCode

* Update CameraSession.kt

* Correctly pass along Frame Processor

* fix: Use AUDIO_BIT_RATE of 16 * 44,1Khz

* Use CAMCORDER instead of MIC microphone

* Use 1 channel

* fix: Use `Orientation`

* Add `native` PixelFormat

* Update iOS to latest Skia integration

* feat: Add `pixelFormat` property to Camera

* Catch error in configureSession

* Fix JPEG format

* Clean up best match finder

* Update CameraDeviceDetails.kt

* Clamp sizes by maximum CamcorderProfile size

* Remove `getAvailableVideoCodecs`

* chore: release 3.0.0-rc.5

* Use maximum video size of RECORD as default

* Update CameraDeviceDetails.kt

* Add a todo

* Add JSON device to issue report

* Prefer `full` devices and flash

* Lock to 30 FPS on Samsung

* Implement Zoom

* Refactor

* Format -> PixelFormat

* fix: Feat `pixelFormat` -> `pixelFormats`

* Update TROUBLESHOOTING.mdx

* Format

* fix: Implement `zoom` for Photo Capture

* fix: Don't run if `isActive` is `false`

* fix: Call `examplePlugin(frame)`

* fix: Fix Flash

* fix: Use `react-native-worklets-core`!

* fix: Fix import
This commit is contained in:
Marc Rousavy
2023-08-21 12:50:14 +02:00
committed by GitHub
parent 61fd4e0474
commit 37a3548a81
141 changed files with 3991 additions and 2251 deletions
Download Patch File Download Diff File Expand all files Collapse all files

58
example/ios/Podfile.lock
View File

@@ -15,6 +15,18 @@ PODS:
- hermes-engine/Pre-built (= 0.72.3)
- hermes-engine/Pre-built (0.72.3)
- libevent (2.1.12)
- libwebp (1.3.1):
- libwebp/demux (= 1.3.1)
- libwebp/mux (= 1.3.1)
- libwebp/sharpyuv (= 1.3.1)
- libwebp/webp (= 1.3.1)
- libwebp/demux (1.3.1):
- libwebp/webp
- libwebp/mux (1.3.1):
- libwebp/demux
- libwebp/sharpyuv (1.3.1)
- libwebp/webp (1.3.1):
- libwebp/sharpyuv
- RCT-Folly (2021.07.22.00):
- boost
- DoubleConversion
@@ -321,7 +333,7 @@ PODS:
- React-Core
- react-native-safe-area-context (4.7.1):
- React-Core
- react-native-skia (0.1.197):
- react-native-skia (0.1.200):
- React
- React-callinvoker
- React-Core
@@ -330,7 +342,7 @@ PODS:
- react-native-video/Video (= 5.2.1)
- react-native-video/Video (5.2.1):
- React-Core
- react-native-worklets (0.1.0):
- react-native-worklets-core (0.1.0):
- React
- React-callinvoker
- React-Core
@@ -444,9 +456,13 @@ PODS:
- React-jsi (= 0.72.3)
- React-logger (= 0.72.3)
- React-perflogger (= 0.72.3)
- RNFastImage (8.6.3):
- React-Core
- SDWebImage (~> 5.11.1)
- SDWebImageWebPCoder (~> 0.8.4)
- RNGestureHandler (2.12.1):
- React-Core
- RNReanimated (3.4.1):
- RNReanimated (3.4.2):
- DoubleConversion
- FBLazyVector
- glog
@@ -475,20 +491,26 @@ PODS:
- React-RCTText
- ReactCommon/turbomodule/core
- Yoga
- RNScreens (3.23.0):
- RNScreens (3.24.0):
- React-Core
- React-RCTImage
- RNStaticSafeAreaInsets (2.2.0):
- React-Core
- RNVectorIcons (10.0.0):
- React-Core
- SDWebImage (5.11.1):
- SDWebImage/Core (= 5.11.1)
- SDWebImage/Core (5.11.1)
- SDWebImageWebPCoder (0.8.5):
- libwebp (~> 1.0)
- SDWebImage/Core (~> 5.10)
- SocketRocket (0.6.1)
- VisionCamera (3.0.0-rc.4):
- React
- React-callinvoker
- React-Core
- react-native-skia
- react-native-worklets
- react-native-worklets-core
- Yoga (1.14.0)
DEPENDENCIES:
@@ -520,7 +542,7 @@ DEPENDENCIES:
- react-native-safe-area-context (from `../node_modules/react-native-safe-area-context`)
- "react-native-skia (from `../node_modules/@shopify/react-native-skia`)"
- react-native-video (from `../node_modules/react-native-video`)
- react-native-worklets (from `../node_modules/react-native-worklets`)
- react-native-worklets-core (from `../node_modules/react-native-worklets-core`)
- React-NativeModulesApple (from `../node_modules/react-native/ReactCommon/react/nativemodule/core/platform/ios`)
- React-perflogger (from `../node_modules/react-native/ReactCommon/reactperflogger`)
- React-RCTActionSheet (from `../node_modules/react-native/Libraries/ActionSheetIOS`)
@@ -538,6 +560,7 @@ DEPENDENCIES:
- React-runtimescheduler (from `../node_modules/react-native/ReactCommon/react/renderer/runtimescheduler`)
- React-utils (from `../node_modules/react-native/ReactCommon/react/utils`)
- ReactCommon/turbomodule/core (from `../node_modules/react-native/ReactCommon`)
- RNFastImage (from `../node_modules/react-native-fast-image`)
- RNGestureHandler (from `../node_modules/react-native-gesture-handler`)
- RNReanimated (from `../node_modules/react-native-reanimated`)
- RNScreens (from `../node_modules/react-native-screens`)
@@ -550,6 +573,9 @@ SPEC REPOS:
trunk:
- fmt
- libevent
- libwebp
- SDWebImage
- SDWebImageWebPCoder
- SocketRocket
EXTERNAL SOURCES:
@@ -606,8 +632,8 @@ EXTERNAL SOURCES:
:path: "../node_modules/@shopify/react-native-skia"
react-native-video:
:path: "../node_modules/react-native-video"
react-native-worklets:
:path: "../node_modules/react-native-worklets"
react-native-worklets-core:
:path: "../node_modules/react-native-worklets-core"
React-NativeModulesApple:
:path: "../node_modules/react-native/ReactCommon/react/nativemodule/core/platform/ios"
React-perflogger:
@@ -642,6 +668,8 @@ EXTERNAL SOURCES:
:path: "../node_modules/react-native/ReactCommon/react/utils"
ReactCommon:
:path: "../node_modules/react-native/ReactCommon"
RNFastImage:
:path: "../node_modules/react-native-fast-image"
RNGestureHandler:
:path: "../node_modules/react-native-gesture-handler"
RNReanimated:
@@ -666,6 +694,7 @@ SPEC CHECKSUMS:
glog: 04b94705f318337d7ead9e6d17c019bd9b1f6b1b
hermes-engine: 10fbd3f62405c41ea07e71973ea61e1878d07322
libevent: 4049cae6c81cdb3654a443be001fb9bdceff7913
libwebp: 33dc822fbbf4503668d09f7885bbfedc76c45e96
RCT-Folly: 424b8c9a7a0b9ab2886ffe9c3b041ef628fd4fb1
RCTRequired: a2faf4bad4e438ca37b2040cb8f7799baa065c18
RCTTypeSafety: cb09f3e4747b6d18331a15eb05271de7441ca0b3
@@ -684,9 +713,9 @@ SPEC CHECKSUMS:
react-native-blur: cfdad7b3c01d725ab62a8a729f42ea463998afa2
react-native-cameraroll: 134805127580aed23403b8c2cb1548920dd77b3a
react-native-safe-area-context: 9697629f7b2cda43cf52169bb7e0767d330648c2
react-native-skia: e2cb3443442bf7680e3276dc82cd87c97f96b6f9
react-native-skia: d0b0aab6bb1f146eb6f379fb671b719deabd20fb
react-native-video: c26780b224543c62d5e1b2a7244a5cd1b50e8253
react-native-worklets: c7576ad4ad0f030ff41e8d74ad0077c96054a6c1
react-native-worklets-core: c7576ad4ad0f030ff41e8d74ad0077c96054a6c1
React-NativeModulesApple: c57f3efe0df288a6532b726ad2d0322a9bf38472
React-perflogger: 6bd153e776e6beed54c56b0847e1220a3ff92ba5
React-RCTActionSheet: c0b62af44e610e69d9a2049a682f5dba4e9dff17
@@ -704,13 +733,16 @@ SPEC CHECKSUMS:
React-runtimescheduler: 837c1bebd2f84572db17698cd702ceaf585b0d9a
React-utils: bcb57da67eec2711f8b353f6e3d33bd8e4b2efa3
ReactCommon: 3ccb8fb14e6b3277e38c73b0ff5e4a1b8db017a9
RNFastImage: 5c9c9fed9c076e521b3f509fe79e790418a544e8
RNGestureHandler: c0d04458598fcb26052494ae23dda8f8f5162b13
RNReanimated: 53ca20eee770c41173703f5948cd8898aa08262c
RNScreens: 6a8a3c6b808aa48dca1780df7b73ea524f602c63
RNReanimated: 726395a2fa2f04cea340274ba57a4e659bc0d9c1
RNScreens: b21dc57dfa2b710c30ec600786a3fc223b1b92e7
RNStaticSafeAreaInsets: 055ddbf5e476321720457cdaeec0ff2ba40ec1b8
RNVectorIcons: 8b5bb0fa61d54cd2020af4f24a51841ce365c7e9
SDWebImage: a7f831e1a65eb5e285e3fb046a23fcfbf08e696d
SDWebImageWebPCoder: 908b83b6adda48effe7667cd2b7f78c897e5111d
SocketRocket: f32cd54efbe0f095c4d7594881e52619cfe80b17
VisionCamera: d0112c5121c8fc785ed9c2a1e4a557ae22088709
VisionCamera: 2ee7d7545925a09d996c4bd70438ebc64714eccc
Yoga: 8796b55dba14d7004f980b54bcc9833ee45b28ce
PODFILE CHECKSUM: ab9c06b18c63e741c04349c0fd630c6d3145081c