* docs: New V3 docs for new API * fix: Prefer Wide-Angle unless explicitly opted-out * docs: Update DEVICES * Finish Devices docs * Switch links * Revert "Switch links" This reverts commit 06f196ae0e67787cbd5768e125be6d0a3cb5bbc9. * docs: New LIFECYCLE * docs: New CAPTURING docs * Update Worklets links * docs: Update TROUBLESHOOTING and ZOOMING * fix: Update `getAvailableCameraDevices()` usages * docs: Update FORMATS * Update Errors.kt * docs: Fix broken links * docs: Update references to old hooks * docs: Create Frame Processor Tips * docs: Auto-dark mode * fix: Fix FPS filter * feat: Add `'max'` flag to format filter * fix: Use loop * fix: Fix bug in `getCameraFormat` * fix: Find best aspect ratio as well * fix: Switch between formats on FPS change * Update FRAME_PROCESSOR_PLUGIN_LIST.mdx * Add FPS graph explanation * feat: Support HDR filter * docs: Add HDR docs * docs: Add Video Stabilization * docs: Update Skia docs * Skia links * Add Skia labels * Update SKIA_FRAME_PROCESSORS.mdx * docs: Add Performance * Update some wording * Update headers / and zoom * Add examples for devices * fix highlights * fix: Expose `Frame` * docs: Update FP docs * Update links * Update FRAME_PROCESSOR_CREATE_PLUGIN_IOS.mdx
138 lines
4.5 KiB
Plaintext
138 lines
4.5 KiB
Plaintext
---
|
|
id: performance
|
|
title: Performance
|
|
sidebar_label: Performance
|
|
---
|
|
|
|
import Tabs from '@theme/Tabs';
|
|
import TabItem from '@theme/TabItem';
|
|
import useBaseUrl from '@docusaurus/useBaseUrl';
|
|
|
|
## Performance of VisionCamera
|
|
|
|
VisionCamera is highly optimized to be **as fast as a native Camera app**, and is sometimes even faster than that.
|
|
I am using highly efficient native GPU buffer formats (such as YUV 4:2:0, or lossless compressed YUV 4:2:0), running the video pipelines in parallel, using C++ for the Frame Processors implementation, and other tricks to make sure VisionCamera is as efficient as possible.
|
|
|
|
## Making it faster
|
|
|
|
There are a few things you can do to make your Camera faster which requires a core understanding of how Cameras work under the hood:
|
|
|
|
### Simpler Camera Device
|
|
|
|
Selecting a "simpler" Camera Device (i.e. a Camera Device with _less physical cameras_) allows the Camera to initialize faster as it does not have to start multiple devices at once.
|
|
You can prefer a simple wide-angle Camera (`['wide-angle-camera']`) over a triple camera (`['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera']`) to significantly speed up initialization time.
|
|
|
|
<Tabs
|
|
groupId="component-style"
|
|
defaultValue="hooks"
|
|
values={[
|
|
{label: 'Hooks API', value: 'hooks'},
|
|
{label: 'Imperative API', value: 'imperative'}
|
|
]}>
|
|
<TabItem value="hooks">
|
|
|
|
```ts
|
|
const fasterDevice = useCameraDevice('back', {
|
|
physicalDevices: ['wide-angle-camera']
|
|
})
|
|
const slowerDevice = useCameraDevice('back', {
|
|
physicalDevices: ['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera']
|
|
})
|
|
```
|
|
|
|
</TabItem>
|
|
<TabItem value="imperative">
|
|
|
|
```ts
|
|
const devices = Camera.getAvailableCameraDevices()
|
|
const fasterDevice = getCameraDevice(devices, 'back', {
|
|
physicalDevices: ['wide-angle-camera']
|
|
})
|
|
const slowerDevice = getCameraDevice(devices, 'back', {
|
|
physicalDevices: ['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera']
|
|
})
|
|
```
|
|
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
See ["Camera Devices"](devices) for more information.
|
|
|
|
Note: By default (when not passing the options object), a simpler device is already chosen.
|
|
|
|
### No HDR
|
|
|
|
HDR uses 10-bit formats and/or additional processing steps that come with additional computation overhead. Disable HDR (don't pass `hdr` to the `<Camera>`) for higher efficiency.
|
|
|
|
### Buffer Compression
|
|
|
|
Enable Buffer Compression ([`enableBufferCompression`](/docs/api/interfaces/CameraProps#enablebuffercompression)) to use lossless-compressed buffers for the Camera's video pipeline. These buffers can use less memory and are more efficient.
|
|
|
|
Note: When not using a `frameProcessor`, buffer compression is automatically enabled.
|
|
|
|
### Video Stabilization
|
|
|
|
Video Stabilization requires additional overhead to start the algorithm, so disabling [`videoStabilizationMode`](/docs/api/interfaces/CameraProps#videoStabilizationMode) can significantly speed up the Camera initialization time.
|
|
|
|
### Pixel Format
|
|
|
|
By default, the `native` [`PixelFormat`](/docs/api#PixelFormat) is used, which is much more efficient than `rgb`.
|
|
|
|
### Disable unneeded pipelines
|
|
|
|
Only enable [`photo`](/docs/api/interfaces/CameraProps#photo) and [`video`](/docs/api/interfaces/CameraProps#video) if needed.
|
|
|
|
### Fast Photos
|
|
|
|
If you need to take photos as fast as possible, use a [`qualityPrioritization`](/docs/api/interfaces/TakePhotoOptions#qualityprioritization) of `'speed'` to speed up the photo pipeline:
|
|
|
|
```ts
|
|
camera.current.takePhoto({
|
|
qualityPrioritization: 'speed'
|
|
})
|
|
```
|
|
|
|
### Appropriate Format resolution
|
|
|
|
Choose formats efficiently. If your backend can only handle 1080p videos, don't select a 4k format if you have to downsize it later anyways - instead use 1080p already for the Camera:
|
|
|
|
<Tabs
|
|
groupId="component-style"
|
|
defaultValue="hooks"
|
|
values={[
|
|
{label: 'Hooks API', value: 'hooks'},
|
|
{label: 'Imperative API', value: 'imperative'}
|
|
]}>
|
|
<TabItem value="hooks">
|
|
|
|
```ts
|
|
const format = useCameraFormat(device, [
|
|
{ videoResolution: { width: 1920, height: 1080 } }
|
|
])
|
|
```
|
|
|
|
</TabItem>
|
|
<TabItem value="imperative">
|
|
|
|
```ts
|
|
const format = getCameraFormat(device, [
|
|
{ videoResolution: { width: 1920, height: 1080 } }
|
|
])
|
|
```
|
|
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
### Appropriate Format FPS
|
|
|
|
Same as with format resolutions, also record at the frame rate you expect. Setting your frame rate higher can use more memory and heat up the battery.
|
|
If your backend can only handle 30 FPS, there is no need to record at 60 FPS, instead set the Camera' [`fps`](/docs/api/interfaces/CameraProps#fps) to 30:
|
|
|
|
```jsx
|
|
return <Camera {...props} fps={30} />
|
|
```
|
|
|
|
<br />
|
|
|
|
#### 🚀 Next section: [Camera Errors](errors)
|