react-native-vision-camera/docs/docs/guides/PERFORMANCE.mdx
Marc Rousavy 1a0bd8f7c2
feat: Add enableGpuBuffers property (#2557)
* Revert "fix: Fix VideoPipeline crash on Samsung (Disable `USAGE_GPU_SAMPLED_IMAGE` ImageReader) (#2555)"

This reverts commit ad33dd91b1.

* feat: Add `enableGpuBuffers` prop

* Create ImageWriter separately
2024-02-14 12:47:40 +01:00

152 lines
5.5 KiB
Plaintext

---
id: performance
title: Performance
sidebar_label: Performance
---
import Tabs from '@theme/Tabs'
import TabItem from '@theme/TabItem'
import useBaseUrl from '@docusaurus/useBaseUrl'
## Performance of VisionCamera
VisionCamera is highly optimized to be **as fast as a native Camera app**, and is sometimes even faster than that.
I am using highly efficient native GPU buffer formats (such as YUV 4:2:0, or lossy compressed YUV 4:2:0), running the video pipelines in parallel, using C++ for the Frame Processors implementation, and other tricks to make sure VisionCamera is as efficient as possible.
## Making it faster
There are a few things you can do to make your Camera faster which requires a core understanding of how Cameras work under the hood:
### Simpler Camera Device
Selecting a "simpler" Camera Device (i.e. a Camera Device with _less physical cameras_) allows the Camera to initialize faster as it does not have to start multiple devices at once.
You can prefer a simple wide-angle Camera (`['wide-angle-camera']`) over a triple camera (`['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera']`) to significantly speed up initialization time.
<Tabs
groupId="component-style"
defaultValue="hooks"
values={[
{label: 'Hooks API', value: 'hooks'},
{label: 'Imperative API', value: 'imperative'}
]}>
<TabItem value="hooks">
```ts
const fasterDevice = useCameraDevice('back', {
physicalDevices: ['wide-angle-camera']
})
const slowerDevice = useCameraDevice('back', {
physicalDevices: ['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera']
})
```
</TabItem>
<TabItem value="imperative">
```ts
const devices = Camera.getAvailableCameraDevices()
const fasterDevice = getCameraDevice(devices, 'back', {
physicalDevices: ['wide-angle-camera']
})
const slowerDevice = getCameraDevice(devices, 'back', {
physicalDevices: ['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera']
})
```
</TabItem>
</Tabs>
See ["Camera Devices"](devices) for more information.
Note: By default (when not passing the options object), a simpler device is already chosen.
### No Video HDR
Video HDR uses 10-bit formats and/or additional processing steps that come with additional computation overhead. Disable [`videoHdr`](/docs/api/interfaces/CameraProps#videohdr) for higher efficiency.
### Buffer Compression
Enable Buffer Compression ([`enableBufferCompression`](/docs/api/interfaces/CameraProps#enablebuffercompression)) to use lossy-compressed buffers for the Camera's video pipeline. These buffers can use less memory and are more efficient.
Note: When not using a `frameProcessor`, buffer compression is automatically enabled.
### GPU buffers
Enable GPU Buffer flags ([`enableGpuBuffers`](/docs/api/interfaces/CameraProps#enablegpubuffers)) to optimize the Video Pipeline for zero-copy buffer forwarding.
If this is enabled, the Video Pipeline can avoid an additional CPU -> GPU copy, resulting in better performance and more efficiency.
Note: This only takes effect when using a `frameProcessor`.
### Video Stabilization
Video Stabilization requires additional overhead to start the algorithm, so disabling [`videoStabilizationMode`](/docs/api/interfaces/CameraProps#videostabilizationmode) can significantly speed up the Camera initialization time.
### Pixel Format
By default, the `native` [`PixelFormat`](/docs/api#pixelformat) is used, which is much more efficient than `rgb`.
- On iOS, `native` is `yuv`
- On Android `native` is some kind of vendor specific format, which might be `yuv`
### Disable unneeded pipelines
Only enable [`photo`](/docs/api/interfaces/CameraProps#photo), [`video`](/docs/api/interfaces/CameraProps#video), [`codeScanner`](/docs/api/interfaces/CameraProps#codescanner) or [`frameProcessor`](/docs/api/interfaces/CameraProps#frameprocessor) if needed.
### Using `isActive`
The [`isActive`](/docs/api/interfaces/CameraProps#isactive) prop controls whether the Camera should actively stream frames. Instead of fully unmounting the `<Camera>` component and remounting it again, keep it mounted and just switch `isActive` on or off. This makes the Camera resume much faster as it internally keeps the session warmed up.
### Fast Photos
If you need to take photos as fast as possible, use a [`qualityPrioritization`](/docs/api/interfaces/TakePhotoOptions#qualityprioritization) of `'speed'` to speed up the photo pipeline:
```ts
camera.current.takePhoto({
qualityPrioritization: 'speed'
})
```
### Appropriate Format resolution
Choose formats efficiently. If your backend can only handle 1080p videos, don't select a 4k format if you have to downsize it later anyways - instead use 1080p already for the Camera:
<Tabs
groupId="component-style"
defaultValue="hooks"
values={[
{label: 'Hooks API', value: 'hooks'},
{label: 'Imperative API', value: 'imperative'}
]}>
<TabItem value="hooks">
```ts
const format = useCameraFormat(device, [
{ videoResolution: { width: 1920, height: 1080 } }
])
```
</TabItem>
<TabItem value="imperative">
```ts
const format = getCameraFormat(device, [
{ videoResolution: { width: 1920, height: 1080 } }
])
```
</TabItem>
</Tabs>
### Appropriate Format FPS
Same as with format resolutions, also record at the frame rate you expect. Setting your frame rate higher can use more memory and heat up the battery.
If your backend can only handle 30 FPS, there is no need to record at 60 FPS, instead set the Camera' [`fps`](/docs/api/interfaces/CameraProps#fps) to 30:
```jsx
return <Camera {...props} fps={30} />
```
<br />
#### 🚀 Next section: [Camera Errors](errors)