Commit Graph

25 Commits

Author SHA1 Message Date
Marc Rousavy
5acc64e031
feat: Create persistent CaptureSession to avoid any blackscreen issues or errors (#2494)
* feat: Create custom `CaptureSession` wrapper

* Create `PersistentCameraCaptureSession`

* Update VideoStabilizationMode.kt

* Create RepeatingRequest.kt

* Update CaptureSession.kt

* Delete CaptureSession.kt

* Update PersistentCameraCaptureSession.kt

* Update PersistentCameraCaptureSession.kt

* fix: Add `isRepeating`

* Update CameraSession.kt

* Make `SurfaceOutput` not `Closable` anymore

* Update PersistentCameraCaptureSession.kt

* Stub out the rest

* Format

* Set `isRunning` properly

* Close previous outputs

* onError callback

* Format

* Started/Stopped

* Update CameraPage.tsx

* Add `isValid`

* Log `isActive`

* Add `tryAbortCaptures`

* Configure()

* Try?

* Add `didDestroyFromOutside`

* Disable FP for testing

* fix: Call `super.onAttachedToWindow` first

* Hm

* Update CameraSession.kt

* Update PersistentCameraCaptureSession.kt

* Try catch `didDestroyFromOutside`

* Update PersistentCameraCaptureSession.kt

* Session can only be active with a preview

* Update PersistentCameraCaptureSession.kt

* Throw `no-outputs` if needed

* Update logs

* fix: Check for CAMERA permission

* fix: Close session when opening a new device

* perf: Make everything `by lazy` in CameraDeviceDetails

* Update CameraDeviceDetails.kt

* Update PersistentCameraCaptureSession.kt

* Update PersistentCameraCaptureSession.kt

* Move

* Update Podfile.lock

* Implement `capture()`

* Format

* fix: Fix orientation not being applied

* fix: Fix `isMirrored`

* fix: Fix getting size

* fix: Close `Surface` in `VideoPipeline`

* Format

* fix: Fix `VideoPipeline` not properly destroying itself

* Use FP again

* Update CameraConfiguration.kt

* Rename

* Clean up

* Format

* Update CameraConfiguration.kt

* fix: Don't stop repeating request when capturing
2024-02-06 14:19:25 +01:00
Marc Rousavy
ae75e22fc0
fix: Fix crash in toArrayBuffer() by properly acquiring a reference on AHardwareBuffer* (#2490)
* fix: Fix crash in `toArrayBuffer()` by properly acquiring a reference on `AHardwareBuffer*`

* Format

* Update Podfile.lock
2024-01-31 20:32:02 +01:00
Marc Rousavy
5f75b9e8dc
fix: Fix HostObject destructors to make sure a JNI Environment is set up (#2462)
* fix: Fix HostObject destructors to make sure a JNI Environment is set up

* Use `reset` instead of `= nullptr`

* Format

* Format
2024-01-30 16:51:09 +01:00
Marc Rousavy
9e1297531e
fix: Use ThreadScope::WithClassLoader to also load custom JNI Types (#2461)
* fix: Fix `ThreadScope` in `runAsync`

* fix: Use `&`

* Update JVisionCameraScheduler.cpp
2024-01-30 16:37:00 +01:00
Marc Rousavy
33e2adcec6
fix: Fix Unable to retrieve jni environment. Is the thread attached? errors by using jni::ThreadScope (#2457)
fix: Add `jni::ThreadScope` to `JVisionCameraScheduler`
2024-01-30 15:39:33 +01:00
Marc Rousavy
c911d25727
fix: Fix SharedArray JSI constructor allocating a new array instead of wrapping (#2444) 2024-01-26 12:05:00 +01:00
Marc Rousavy
bc6f954bf1
fix: Fix Android C++ SharedArray logs 2024-01-25 16:34:45 +01:00
Marc Rousavy
ba1d7eec9c
feat: Use JSI's ArrayBuffer instead of TypedArray (#2408)
* feat: Use JSI's `ArrayBuffer` instead of `TypedArray`

* fix: Fix move memory

* feat: Implement iOS

* Format

* Update JSIJNIConversion.cpp

* fix: Fix Android `toArrayBuffer` and other

* Catch FP call errors

* Update return type

* Use `CPU_READ_OFTEN` flag as well

* CPU flag

* Run destructors under `jni::ThreadScope`

* Update FrameProcessorPluginHostObject.cpp

* fix: Fix `toArrayBuffer()` crash

* Update Frame.ts
2024-01-17 20:18:46 +01:00
Marc Rousavy
2f21609e39
feat: Add SharedArray.getSize() (#2406)
* feat: Add `SharedArray.getSize()`

* Rename `count` to `size` on iOS

* `->` instead of `.`
2024-01-17 18:30:26 +01:00
Marc Rousavy
d747933892
fix: Fix SharedArray name 2024-01-12 19:25:13 +01:00
Marc Rousavy
fe614a4c21
fix: Fix JNI TypedArray error 2024-01-12 19:24:46 +01:00
Marc Rousavy
29fe98cc44
feat: Add zero-copy SharedArray type to Frame Processor Plugins (#2383)
* feat: Create `TypedArray` class for Frame Processor Plugins

* Type

* feat: Pass `VisionCameraProxy` along (BREAKING)

* feat: Finish implementation

* Log a bit

* feat: Successfully convert JSI <> JNI buffers

* Wrap buffer

* fix: Fix using wrong Runtime

* feat: Add docs

* add zero copy example

* Format C++

* Create iOS base

* feat: Finish iOS implementation

* chore: Format

* fix: Use `NSData` instead of `NSMutableData`

* Format

* fix: Fix build when Frame Processors are disabled

* chore: Rename `TypedArray` to `SharedArray`

* fix: Fix Swift typings for Array

* Remove a few default inits

* fix: Fix Android build

* fix: Use `NSInteger`

* Update SharedArray.mm

* fix: Expose bytes directly on iOS (NSData was immutable)
2024-01-12 16:00:36 +01:00
Marc Rousavy
34c5b11927
feat: Re-throw error on JS side instead of just logging on native side (#2366)
* feat: Re-throw error on JS side instead of just logging on native side

* fix: Fix proxy

* fix: Fix app crash by only logging error

* fix: Use `global.ErrorUtils` (from reanimated)
2024-01-11 17:23:38 +01:00
Marc Rousavy
ef4e9fabf9
fix: Unwrap boxed JNI types directly (#2337)
* fix: Unwrap boxed JNI types directly

* Update JSIJNIConversion.cpp
2024-01-02 14:13:47 +01:00
Marc Rousavy
3dc75112ae
fix: Fix crash when trying to console.log(frame) (#2335)
* feat: Override `toJSON` to support `console.log(frame)`

* Update FrameHostObject.mm

* Also use on Android

* Update FrameHostObject.mm
2024-01-02 13:54:07 +01:00
Marc Rousavy
895f3ec889
feat: Make Frame thread-safe and improve error messages (#2327)
* fix: Fix multi-Thread access on Java

* fix: Thread-lock access on iOS as well

* whoops add missing header impl

* Update Podfile.lock

* fix: Don't use `CFGetRetainCount`

* fix: Lock access on iOS as well

* C++ format

* More detailed error

* chore: Move getters into `Frame`

* Format c++

* Use enum `orientation` again

* format

* fix: Synchronize `isValid` on Java

* Also log pixelformat

* feat: Use Java enums in C++

* Format C++
2023-12-29 14:09:56 +01:00
Marc Rousavy
07027d8010
fix: Rename getFrameProcessorPlugin to initFrameProcessorPlugin (#2038)
* fix: Rename `getFrameProcessorPlugin` to `initFrameProcessorPlugin`

* fix: Make nullable, add comments

* Format

* Update FrameProcessorPlugin.java

* Update FrameProcessorPlugin.h

* fix: Fix dead links

* Call super constructor

* Update ExampleFrameProcessorPlugin.java

* fix: Init calls
2023-10-19 11:19:47 +02:00
Marc Rousavy
8d90dcc476
fix: Make HardwareBuffers compile optionally (opt-out in minSdkVersion <26) (#1910)
* fix: Make HardwareBuffers compile optionally

* chore: Format C++ code

* fix: Fix if
2023-10-03 13:18:41 +02:00
Marc Rousavy
cf1952d8f7
feat: Use AHardwareBuffer* for frame.toArrayBuffer() (#1888)
* feat: Route images through `ImageWriter` into OpenGL pipeline

* fix: Use RGB format

* fix: Every device supports YUV, RGB and NATIVE

* Update VideoPipeline.kt

* log format

* Plug ImageReader between OpenGL pipeline

* Call Frame Processor

* Format

* Remove logs

* feat: Use `HardwareBuffer` for `toArrayBuffer()`

* Format
2023-09-29 21:54:04 +02:00
Marc Rousavy
954b44810b
feat: Route images through ImageWriter into OpenGL pipeline (#1874)
* feat: Route images through `ImageWriter` into OpenGL pipeline

* fix: Use RGB format

* fix: Every device supports YUV, RGB and NATIVE

* Update VideoPipeline.kt

* log format

* Plug ImageReader between OpenGL pipeline

* Call Frame Processor

* Format

* Remove logs
2023-09-29 21:52:19 +02:00
Marc Rousavy
9add0eb571
feat: Use C++ OpenGL GPU VideoPipeline again (#1836)
1. Reverts 4e96eb77e0 (PR #1789) to bring the C++ OpenGL GPU Pipeline back.
2. Fixes the "initHybrid JNI not found" error by loading the native JNI/C++ library in `VideoPipeline.kt`.

This PR has two downsides:

1. `pixelFormat="yuv"` does not work on Android. OpenGL only works in RGB
2. OpenGL rendering is fast, but it has an overhead. I think for Camera -> Video Recording we shouldn't be using an entire OpenGL rendering pipeline.

The original plan was to use something similar to how it works on iOS by just passing GPU buffers around, but the android.media APIs just aren't as advanced yet. `ImageReader`/`ImageWriter` is way too buggy and doesn't really work with `MediaRecorder`/`MediaCodec`.

This sucks, I hope in the future we can use something like `AHardwareBuffer`s.
2023-09-22 17:22:31 +02:00
Marc Rousavy
4e96eb77e0
perf: Use ImageWriter instead of OpenGL Pipeline for faster processing (#1789)
* perf: Use `ImageWriter` instead of OpenGL Pipeline for faster processing

* chore: Remove C++ part of OpenGL pipeline

* Clean up

* Update README.md
2023-09-11 13:58:58 +02:00
Marc Rousavy
b4b0e49eb5
fix: Also properly convert any[] -> List<Object> in FP Android (#1760)
* fix: Also properly convert `any[]` -> `List<Object>` in FP Android

* Format C++

* fix: Add missing return
2023-09-06 10:56:33 +02:00
Marc Rousavy
325fc4d2bb chore: Change line width to 140 in C++ 2023-09-01 19:39:25 +02:00
Marc Rousavy
036856aed5
chore: Move everything into package/ (#1745)
* Move everything into package

* Remove .DS_Store

* Move scripts and eslintrc to package

* Create CODE_OF_CONDUCT.md

* fix some links

* Update all links (I think)

* Update generated docs

* Update notice-yarn-changes.yml

* Update validate-android.yml

* Update validate-cpp.yml

* Delete notice-yarn-changes.yml

* Update validate-cpp.yml

* Update validate-cpp.yml

* Update validate-js.yml

* Update validate-cpp.yml

* Update validate-cpp.yml

* wrong c++ style

* Revert "wrong c++ style"

This reverts commit 55a3575589c6f13f8b05134d83384f55e0601ab2.
2023-09-01 18:15:28 +02:00