* Revert "feat: Skia for Android (#1731)"
This reverts commit a7c137da07.
* Remove some skia
* Remove all the Skia stuff.
* Update useFrameProcessor.ts
* Update lockfiles
* fix: Use native Preview again
* Use `OpenGLTexture&` again
* Remove `PreviewOutput` (we use `SurfaceView` in parallel)
* fix: Log photo widths
* fix: Fix cpplint
* feat: Call Skia Renderer
* Use default NativePreviewView for Skia
* Render to separate FBO
* It appears once
* Refactor a lot lol
* Pass width/height
* Read width/heights
* Update SkiaRenderer.cpp
* Read stencil/samples
* Use switch for target
* Clear full red
* Update VideoPipeline.cpp
* fix: Use `BorrowTextureFrom` instead of `AdoptTextureFrom`
* Get it to work
* Draw Camera Frame again (only works for first frame)
* glDisable(GL_BLEND)
* Use Frame Buffer again
* Simplify Skia offscreen surface creation
* fix: Get it to kinda work?
* fix: Remove `sampler2D` shader
Only the EXTERNAL_OES one kinda works
* Revert "fix: Remove `sampler2D` shader"
This reverts commit bf241a82f440f5a442f23a2b10329b813e7cdb3e.
* Revert "fix: Get it to kinda work?"
This reverts commit ea6a8784ad8dc7d05e8076591874f021b51dd84a.
* fix: Use Skia for rendering
* Simplify drawing code a lot
* Clean up drawing loop a bit more
* Some docs
* Update SkiaRenderer.cpp
* Surface
* try to use Matrix
* Use BottomLeft as a surface origin again
* Get actual surface dimensions
* Use 1x1 pbuffer instead
* Update SkiaRenderer.cpp
* Update SkiaRenderer.cpp
* feat: Implement Skia Frame Processor (#1735)
* feat: Implement JS Skia Frame Processor
* Update SkiaRenderer.cpp
* push
* Create Frame from C++
* compile
* Compile
* Update VideoPipeline.cpp
* Fix JNI local ref
* Use `HardwareBuffer` for implementation
* feat: Custom `Frame` implementation that uses CPU `ByteBuffer` (#1736)
* feat: Implement JS Skia Frame Processor
* Update SkiaRenderer.cpp
* push
* Create Frame from C++
* compile
* Compile
* Update VideoPipeline.cpp
* Fix JNI local ref
* Use `HardwareBuffer` for implementation
* try: Try to just create a CPU based ByteBuffer
* fix: Fix Java Type
* fix remaining errors
* try fixing FrameFactory
* Use `free`
* fix: Fix scene mode crash on some emulators
* fix: Fix scene mode crash on some emulators
* Fix getting pixels
* fix: Fix buffer not being freed
* Add some docs to `Frame`
* Test Skia again
* Use `getCurrentPresentationTime()`
* Remove `FrameFactory.cpp`
* Update VideoPipeline.h
* Update VideoPipeline.cpp
* Create `VideoPipeline` c++
* Remove folly C++ dependency
* Create `VideoPipeline` HybridClass
* Set up OpenGL
* Add outputs
* Update VideoPipeline.kt
* Bum `minSdkVersion` to `26`
* Create `VideoPipelineOutput`
* Create output funcs
* Set output pipelines
* Add FP/Recording on Output change
* Update VideoPipeline.cpp
* Create `PassThroughShader`
* Try to draw? I have honestly no idea
* fix: Fix `setFrameProcessor` nameclash
* fix: Fix `high-res-sizes` being null
* Add preview output
* Create `OpenGLContext.cpp`
* Make screen red
* This _should_ work (MESSY)
* FINALLY RENDER TEXTURE
* Rotate
* Mirror
* Clean up a bit
* Add `getWidth()`/`getHeight()`
* Cleanup
* fix: Use uniforms instead of attributes
* Draw with passed rotation/mirror mode
* feat: Use SurfaceTexture's transformMatrix in OpenGL pipeline (#1727)
* feat: Use Transform Matrix from SurfaceTexture
* Renam
* feat: Fix OpenGL Shader
* Update VideoPipeline.kt
* Measure elapsed time
* fix: Fix low resolution
* Render to offscreen
* Render to every context
* Release `SurfaceTexture` on close
* Use one OpenGL context to render to multiple EGLSurfaces
* Clean up a bit
* fix: Fix recording pipeline not triggering
* fix: Synchronize close to prevent nulls
* Update OpenGLRenderer.cpp
* fix: Hardcode Android recorder size
Replaces `ReadableNativeMap`/`WritableNativeMap` with `Map<String, Object>` and `ReadableNativeArray`/`WritableNativeArray` with `List<Object>`, making the JSI -> JNI conversion a bit faster and more logical.
Also, we could now convert Array Buffers or HostObjects if we wanted to.
* Nuke CameraX
* fix: Run View Finder on UI Thread
* Open Camera, set up Threads
* fix init
* Mirror if needed
* Try PreviewView
* Use max resolution
* Add `hardwareLevel` property
* Check if output type is supported
* Replace `frameRateRanges` with `minFps` and `maxFps`
* Remove `isHighestPhotoQualitySupported`
* Remove `colorSpace`
The native platforms will use the best / most accurate colorSpace by default anyways.
* HDR
* Check from format
* fix
* Remove `supportsParallelVideoProcessing`
* Correctly return video/photo sizes on Android now. Finally
* Log all Device props
* Log if optimized usecase is used
* Cleanup
* Configure Camera Input only once
* Revert "Configure Camera Input only once"
This reverts commit 0fd6c03f54c7566cb5592053720c4a8743aba92e.
* Extract Camera configuration
* Try to reconfigure all
* Hook based
* Properly set up `CameraSession`
* Delete unused
* fix: Fix recreate when outputs change
* Update NativePreviewView.kt
* Use callback for closing
* Catch CameraAccessException
* Finally got it stable
* Remove isMirrored
* Implement `takePhoto()`
* Add ExifInterface library
* Run findViewById on UI Thread
* Add Photo Output Surface to takePhoto
* Fix Video Stabilization Modes
* Optimize Imports
* More logs
* Update CameraSession.kt
* Close Image
* Use separate Executor in CameraQueue
* Delete hooks
* Use same Thread again
* If opened, call error
* Update CameraSession.kt
* Log HW level
* fix: Don't enable Stream Use Case if it's not 100% supported
* Move some stuff
* Cleanup PhotoOutputSynchronizer
* Try just open in suspend fun
* Some synchronization fixes
* fix logs
* Update CameraDevice+createCaptureSession.kt
* Update CameraDevice+createCaptureSession.kt
* fixes
* fix: Use Snapshot Template for speed capture prio
* Use PREVIEW template for repeating request
* Use `TEMPLATE_RECORD` if video use-case is attached
* Use `isRunning` flag
* Recreate session everytime on active/inactive
* Lazily get values in capture session
* Stability
* Rebuild session if outputs change
* Set `didOutputsChange` back to false
* Capture first in lock
* Try
* kinda fix it? idk
* fix: Keep Outputs
* Refactor into single method
* Update CameraView.kt
* Use Enums for type safety
* Implement Orientation (I think)
* Move RefCount management to Java (Frame)
* Don't crash when dropping a Frame
* Prefer Devices with higher max resolution
* Prefer multi-cams
* Use FastImage for Media Page
* Return orientation in takePhoto()
* Load orientation from EXIF Data
* Add `isMirrored` props and documentation for PhotoFile
* fix: Return `not-determined` on Android
* Update CameraViewModule.kt
* chore: Upgrade packages
* fix: Fix Metro Config
* Cleanup config
* Properly mirror Images on save
* Prepare MediaRecorder
* Start/Stop MediaRecorder
* Remove `takeSnapshot()`
It no longer works on Android and never worked on iOS. Users could use useFrameProcessor to take a Snapshot
* Use `MediaCodec`
* Move to `VideoRecording` class
* Cleanup Snapshot
* Create `SkiaPreviewView` hybrid class
* Create OpenGL context
* Create `SkiaPreviewView`
* Fix texture creation missing context
* Draw red frame
* Somehow get it working
* Add Skia CMake setup
* Start looping
* Init OpenGL
* Refactor into `SkiaRenderer`
* Cleanup PreviewSize
* Set up
* Only re-render UI if there is a new Frame
* Preview
* Fix init
* Try rendering Preview
* Update SkiaPreviewView.kt
* Log version
* Try using Skia (fail)
* Drawwwww!!!!!!!!!! 🎉
* Use Preview Size
* Clear first
* Refactor into SkiaRenderer
* Add `previewType: "none"` on iOS
* Simplify a lot
* Draw Camera? For some reason? I have no idea anymore
* Fix OpenGL errors
* Got it kinda working again?
* Actually draw Frame woah
* Clean up code
* Cleanup
* Update on main
* Synchronize render calls
* holy shit
* Update SkiaRenderer.cpp
* Update SkiaRenderer.cpp
* Refactor
* Update SkiaRenderer.cpp
* Check for `NO_INPUT_TEXTURE`^
* Post & Wait
* Set input size
* Add Video back again
* Allow session without preview
* Convert JPEG to byte[]
* feat: Use `ImageReader` and use YUV Image Buffers in Skia Context (#1689)
* Try to pass YUV Buffers as Pixmaps
* Create pixmap!
* Clean up
* Render to preview
* Only render if we have an output surface
* Update SkiaRenderer.cpp
* Fix Y+U+V sampling code
* Cleanup
* Fix Semaphore 0
* Use 4:2:0 YUV again idk
* Update SkiaRenderer.h
* Set minSdk to 26
* Set surface
* Revert "Set minSdk to 26"
This reverts commit c4085b7c16c628532e5c2d68cf7ed11c751d0b48.
* Set previewType
* feat: Video Recording with Camera2 (#1691)
* Rename
* Update CameraSession.kt
* Use `SurfaceHolder` instead of `SurfaceView` for output
* Update CameraOutputs.kt
* Update CameraSession.kt
* fix: Fix crash when Preview is null
* Check if snapshot capture is supported
* Update RecordingSession.kt
* S
* Use `MediaRecorder`
* Make audio optional
* Add Torch
* Output duration
* Update RecordingSession.kt
* Start RecordingSession
* logs
* More log
* Base for preparing pass-through Recording
* Use `ImageWriter` to append Images to the Recording Surface
* Stream PRIVATE GPU_SAMPLED_IMAGE Images
* Add flags
* Close session on stop
* Allow customizing `videoCodec` and `fileType`
* Enable Torch
* Fix Torch Mode
* Fix comparing outputs with hashCode
* Update CameraSession.kt
* Correctly pass along Frame Processor
* fix: Use AUDIO_BIT_RATE of 16 * 44,1Khz
* Use CAMCORDER instead of MIC microphone
* Use 1 channel
* fix: Use `Orientation`
* Add `native` PixelFormat
* Update iOS to latest Skia integration
* feat: Add `pixelFormat` property to Camera
* Catch error in configureSession
* Fix JPEG format
* Clean up best match finder
* Update CameraDeviceDetails.kt
* Clamp sizes by maximum CamcorderProfile size
* Remove `getAvailableVideoCodecs`
* chore: release 3.0.0-rc.5
* Use maximum video size of RECORD as default
* Update CameraDeviceDetails.kt
* Add a todo
* Add JSON device to issue report
* Prefer `full` devices and flash
* Lock to 30 FPS on Samsung
* Implement Zoom
* Refactor
* Format -> PixelFormat
* fix: Feat `pixelFormat` -> `pixelFormats`
* Update TROUBLESHOOTING.mdx
* Format
* fix: Implement `zoom` for Photo Capture
* fix: Don't run if `isActive` is `false`
* fix: Call `examplePlugin(frame)`
* fix: Fix Flash
* fix: Use `react-native-worklets-core`!
* fix: Fix import
* fix: Fix CI for "Build Android"
* update versions
* Update Gemfile.lock
* format swift
* fix: Fix swift lint
* Update .swiftlint.yml
* Use C++17 for lint
* fix: Fix C++ lints
Before, Frame Processors ran on a separate Thread.
After, Frame Processors run fully synchronous and always at the same FPS as the Camera.
Two new functions have been introduced:
* `runAtTargetFps(fps: number, func: () => void)`: Runs the given code as often as the given `fps`, effectively throttling it's calls.
* `runAsync(frame: Frame, func: () => void)`: Runs the given function on a separate Thread for Frame Processing. A strong reference to the Frame is held as long as the function takes to execute.
You can use `runAtTargetFps` to throttle calls to a specific API (e.g. if your Camera is running at 60 FPS, but you only want to run face detection at ~25 FPS, use `runAtTargetFps(25, ...)`.)
You can use `runAsync` to run a heavy algorithm asynchronous, so that the Camera is not blocked while your algorithm runs. This is useful if your main sync processor draws something, and your async processor is doing some image analysis on the side.
You can also combine both functions.
Examples:
```js
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
console.log("I'm running at 60 FPS!")
}, [])
```
```js
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
console.log("I'm running at 60 FPS!")
runAtTargetFps(10, () => {
'worklet'
console.log("I'm running at 10 FPS!")
})
}, [])
```
```js
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
console.log("I'm running at 60 FPS!")
runAsync(frame, () => {
'worklet'
console.log("I'm running on another Thread, I can block for longer!")
})
}, [])
```
```js
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
console.log("I'm running at 60 FPS!")
runAtTargetFps(10, () => {
'worklet'
runAsync(frame, () => {
'worklet'
console.log("I'm running on another Thread at 10 FPS, I can block for longer!")
})
})
}, [])
```
