1a0bd8f7c2
* Revert "fix: Fix VideoPipeline crash on Samsung (Disable `USAGE_GPU_SAMPLED_IMAGE` ImageReader) (#2555)"
This reverts commit ad33dd91b1
.
* feat: Add `enableGpuBuffers` prop
* Create ImageWriter separately
152 lines
5.5 KiB
Plaintext
152 lines
5.5 KiB
Plaintext
---
|
|
id: performance
|
|
title: Performance
|
|
sidebar_label: Performance
|
|
---
|
|
|
|
import Tabs from '@theme/Tabs'
|
|
import TabItem from '@theme/TabItem'
|
|
import useBaseUrl from '@docusaurus/useBaseUrl'
|
|
|
|
## Performance of VisionCamera
|
|
|
|
VisionCamera is highly optimized to be **as fast as a native Camera app**, and is sometimes even faster than that.
|
|
I am using highly efficient native GPU buffer formats (such as YUV 4:2:0, or lossy compressed YUV 4:2:0), running the video pipelines in parallel, using C++ for the Frame Processors implementation, and other tricks to make sure VisionCamera is as efficient as possible.
|
|
|
|
## Making it faster
|
|
|
|
There are a few things you can do to make your Camera faster which requires a core understanding of how Cameras work under the hood:
|
|
|
|
### Simpler Camera Device
|
|
|
|
Selecting a "simpler" Camera Device (i.e. a Camera Device with _less physical cameras_) allows the Camera to initialize faster as it does not have to start multiple devices at once.
|
|
You can prefer a simple wide-angle Camera (`['wide-angle-camera']`) over a triple camera (`['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera']`) to significantly speed up initialization time.
|
|
|
|
<Tabs
|
|
groupId="component-style"
|
|
defaultValue="hooks"
|
|
values={[
|
|
{label: 'Hooks API', value: 'hooks'},
|
|
{label: 'Imperative API', value: 'imperative'}
|
|
]}>
|
|
<TabItem value="hooks">
|
|
|
|
```ts
|
|
const fasterDevice = useCameraDevice('back', {
|
|
physicalDevices: ['wide-angle-camera']
|
|
})
|
|
const slowerDevice = useCameraDevice('back', {
|
|
physicalDevices: ['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera']
|
|
})
|
|
```
|
|
|
|
</TabItem>
|
|
<TabItem value="imperative">
|
|
|
|
```ts
|
|
const devices = Camera.getAvailableCameraDevices()
|
|
const fasterDevice = getCameraDevice(devices, 'back', {
|
|
physicalDevices: ['wide-angle-camera']
|
|
})
|
|
const slowerDevice = getCameraDevice(devices, 'back', {
|
|
physicalDevices: ['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera']
|
|
})
|
|
```
|
|
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
See ["Camera Devices"](devices) for more information.
|
|
|
|
Note: By default (when not passing the options object), a simpler device is already chosen.
|
|
|
|
### No Video HDR
|
|
|
|
Video HDR uses 10-bit formats and/or additional processing steps that come with additional computation overhead. Disable [`videoHdr`](/docs/api/interfaces/CameraProps#videohdr) for higher efficiency.
|
|
|
|
### Buffer Compression
|
|
|
|
Enable Buffer Compression ([`enableBufferCompression`](/docs/api/interfaces/CameraProps#enablebuffercompression)) to use lossy-compressed buffers for the Camera's video pipeline. These buffers can use less memory and are more efficient.
|
|
|
|
Note: When not using a `frameProcessor`, buffer compression is automatically enabled.
|
|
|
|
### GPU buffers
|
|
|
|
Enable GPU Buffer flags ([`enableGpuBuffers`](/docs/api/interfaces/CameraProps#enablegpubuffers)) to optimize the Video Pipeline for zero-copy buffer forwarding.
|
|
If this is enabled, the Video Pipeline can avoid an additional CPU -> GPU copy, resulting in better performance and more efficiency.
|
|
|
|
Note: This only takes effect when using a `frameProcessor`.
|
|
|
|
### Video Stabilization
|
|
|
|
Video Stabilization requires additional overhead to start the algorithm, so disabling [`videoStabilizationMode`](/docs/api/interfaces/CameraProps#videostabilizationmode) can significantly speed up the Camera initialization time.
|
|
|
|
### Pixel Format
|
|
|
|
By default, the `native` [`PixelFormat`](/docs/api#pixelformat) is used, which is much more efficient than `rgb`.
|
|
|
|
- On iOS, `native` is `yuv`
|
|
- On Android `native` is some kind of vendor specific format, which might be `yuv`
|
|
|
|
### Disable unneeded pipelines
|
|
|
|
Only enable [`photo`](/docs/api/interfaces/CameraProps#photo), [`video`](/docs/api/interfaces/CameraProps#video), [`codeScanner`](/docs/api/interfaces/CameraProps#codescanner) or [`frameProcessor`](/docs/api/interfaces/CameraProps#frameprocessor) if needed.
|
|
|
|
### Using `isActive`
|
|
|
|
The [`isActive`](/docs/api/interfaces/CameraProps#isactive) prop controls whether the Camera should actively stream frames. Instead of fully unmounting the `<Camera>` component and remounting it again, keep it mounted and just switch `isActive` on or off. This makes the Camera resume much faster as it internally keeps the session warmed up.
|
|
|
|
### Fast Photos
|
|
|
|
If you need to take photos as fast as possible, use a [`qualityPrioritization`](/docs/api/interfaces/TakePhotoOptions#qualityprioritization) of `'speed'` to speed up the photo pipeline:
|
|
|
|
```ts
|
|
camera.current.takePhoto({
|
|
qualityPrioritization: 'speed'
|
|
})
|
|
```
|
|
|
|
### Appropriate Format resolution
|
|
|
|
Choose formats efficiently. If your backend can only handle 1080p videos, don't select a 4k format if you have to downsize it later anyways - instead use 1080p already for the Camera:
|
|
|
|
<Tabs
|
|
groupId="component-style"
|
|
defaultValue="hooks"
|
|
values={[
|
|
{label: 'Hooks API', value: 'hooks'},
|
|
{label: 'Imperative API', value: 'imperative'}
|
|
]}>
|
|
<TabItem value="hooks">
|
|
|
|
```ts
|
|
const format = useCameraFormat(device, [
|
|
{ videoResolution: { width: 1920, height: 1080 } }
|
|
])
|
|
```
|
|
|
|
</TabItem>
|
|
<TabItem value="imperative">
|
|
|
|
```ts
|
|
const format = getCameraFormat(device, [
|
|
{ videoResolution: { width: 1920, height: 1080 } }
|
|
])
|
|
```
|
|
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
### Appropriate Format FPS
|
|
|
|
Same as with format resolutions, also record at the frame rate you expect. Setting your frame rate higher can use more memory and heat up the battery.
|
|
If your backend can only handle 30 FPS, there is no need to record at 60 FPS, instead set the Camera' [`fps`](/docs/api/interfaces/CameraProps#fps) to 30:
|
|
|
|
```jsx
|
|
return <Camera {...props} fps={30} />
|
|
```
|
|
|
|
<br />
|
|
|
|
#### 🚀 Next section: [Camera Errors](errors)
|