236 lines
9.5 KiB
Plaintext
236 lines
9.5 KiB
Plaintext
---
|
|
id: devices
|
|
title: Camera Devices
|
|
sidebar_label: Camera Devices
|
|
---
|
|
|
|
import Tabs from '@theme/Tabs'
|
|
import TabItem from '@theme/TabItem'
|
|
import useBaseUrl from '@docusaurus/useBaseUrl'
|
|
|
|
<div class="image-container">
|
|
<svg xmlns="http://www.w3.org/2000/svg" width="283" height="535">
|
|
<image href={useBaseUrl("img/demo.gif")} x="18" y="33" width="247" height="469" />
|
|
<image href={useBaseUrl("img/frame.png")} width="283" height="535" />
|
|
</svg>
|
|
</div>
|
|
|
|
## What are Camera Devices?
|
|
|
|
Camera Devices are the physical (or "virtual") devices that can be used to record videos or capture photos.
|
|
|
|
* **Physical**: A physical Camera Device is a **camera lens on your phone**. Different physical Camera Devices have different specifications, such as different capture formats, resolutions, zoom levels, and more. Some phones have multiple physical Camera Devices.
|
|
|
|
> Examples: _"Backside Wide-Angle Camera"_, _"Frontside Wide-Angle Camera (FaceTime HD)"_, _"Ultra-Wide-Angle back camera"_
|
|
* **Virtual**: A virtual camera device is a **combination of one or more physical camera devices**, and provides features such as _virtual-device-switchover_ while zooming (see video on the right) or _combined photo delivery_ from all physical cameras to produce higher quality images.
|
|
|
|
> Examples: _"Triple-Camera"_, _"Dual-Wide-Angle Camera"_
|
|
|
|
## Select the default Camera
|
|
|
|
If you simply want to use the default [`CameraDevice`](/docs/api/interfaces/CameraDevice), you can just use whatever is available:
|
|
|
|
<Tabs
|
|
groupId="component-style"
|
|
defaultValue="hooks"
|
|
values={[
|
|
{label: 'Hooks API', value: 'hooks'},
|
|
{label: 'Imperative API', value: 'imperative'}
|
|
]}>
|
|
<TabItem value="hooks">
|
|
|
|
```ts
|
|
const device = useCameraDevice('back')
|
|
```
|
|
|
|
</TabItem>
|
|
<TabItem value="imperative">
|
|
|
|
```ts
|
|
const devices = Camera.getAvailableCameraDevices()
|
|
const device = devices.find((d) => d.position === 'back')
|
|
```
|
|
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
And VisionCamera will automatically find the best matching [`CameraDevice`](/docs/api/interfaces/CameraDevice) for you!
|
|
|
|
**🚀 Continue with: [Camera Lifecycle](lifecycle)**
|
|
|
|
## Custom Device Selection
|
|
|
|
For advanced use-cases, you might want to select a different [`CameraDevice`](/docs/api/interfaces/CameraDevice) for your app.
|
|
|
|
A [`CameraDevice`](/docs/api/interfaces/CameraDevice) consists of the following specifications:
|
|
|
|
- [`id`](/docs/api/interfaces/CameraDevice#id): A unique ID used to identify this Camera Device
|
|
- [`position`](/docs/api/interfaces/CameraDevice#position): The position of this Camera Device relative to the phone
|
|
- `back`: The Camera Device is located on the back of the phone
|
|
- `front`: The Camera Device is located on the front of the phone
|
|
- `external`: The Camera Device is an external device. These devices can be either:
|
|
- USB Camera Devices (if they support the [USB Video Class (UVC) Specification](https://en.wikipedia.org/wiki/List_of_USB_video_class_devices))
|
|
- [Continuity Camera Devices](https://support.apple.com/en-us/HT213244) (e.g. your iPhone's or Mac's Camera connected through WiFi/Continuity)
|
|
- Bluetooth/WiFi Camera Devices (if they are supported in the platform-native Camera APIs)
|
|
- [`physicalDevices`](/docs/api/interfaces/CameraDevice#physicaldevices): The physical Camera Devices (lenses) this Camera Device consists of. This can either be one of these values ("physical" device) or any combination of these values ("virtual" device):
|
|
- `ultra-wide-angle-camera`: The "fish-eye" camera for 0.5x zoom
|
|
- `wide-angle-camera`: The "default" camera for 1x zoom
|
|
- `telephoto-camera`: A zoomed-in camera for 3x zoom
|
|
- [`sensorOrientation`](/docs/api/interfaces/CameraDevice#sensororientation): The orientation of the Camera sensor/lens relative to the phone. Cameras are usually in `landscapeLeft` orientation, meaning they are rotated by 90°. This includes their resolutions, so a 4k format might be 3840x2160, not 2160x3840
|
|
- [`minZoom`](/docs/api/interfaces/CameraDevice#minzoom): The minimum possible zoom factor for this Camera Device. If this is a multi-cam, this is the point where the device with the widest field of view is used (e.g. ultra-wide)
|
|
- [`maxZoom`](/docs/api/interfaces/CameraDevice#maxzoom): The maximum possible zoom factor for this Camera Device. If this is a multi-cam, this is the point where the device with the lowest field of view is used (e.g. telephoto)
|
|
- [`neutralZoom`](/docs/api/interfaces/CameraDevice#neutralzoom): A value between `minZoom` and `maxZoom` where the "default" Camera Device is used (e.g. wide-angle). When using multi-cams, make sure to start off at this zoom level, so the user can optionally zoom out to the ultra-wide-angle Camera instead of already starting zoomed out
|
|
- [`formats`](/docs/api/interfaces/CameraDevice#formats): The list of [`CameraDeviceFormat`s](/docs/api/interfaces/CameraDeviceFormat) (See ["Camera Formats"](/docs/guides/formats)) this Camera Device supports. A format specifies:
|
|
- Video Resolution (see ["Formats: Video Resolution"](/docs/guides/formats#video-resolution))
|
|
- Photo Resolution (see ["Formats: Photo Resolution"](/docs/guides/formats#photo-resolution))
|
|
- FPS (see ["Formats: FPS"](/docs/guides/formats#fps))
|
|
- Video Stabilization Mode (see: ["Formats: Video Stabilization Mode"](/docs/guides/formats#videostabilization))
|
|
- Pixel Format (see: ["Formats: Pixel Format"](/docs/guides/formats#pixelformat))
|
|
|
|
### Examples on an iPhone
|
|
|
|
Here's a list of some Camera Devices an iPhone 13 Pro has:
|
|
|
|
- Back Wide Angle Camera (`['wide-angle-camera']`)
|
|
- Back Ultra-Wide Angle Camera (`['ultra-wide-angle-camera']`)
|
|
- Back Telephoto Camera (`['telephoto-camera']`)
|
|
- Back Dual Camera (Wide + Telephoto)
|
|
- Back Dual-Wide Camera (Ultra-Wide + Wide)
|
|
- Back Triple Camera (Ultra-Wide + Wide + Telephoto)
|
|
- Back LiDAR Camera (Wide + LiDAR-Depth)
|
|
- Front Wide Angle (`['wide-angle-camera']`)
|
|
- Front True-Depth (Wide + Depth)
|
|
|
|
### Selecting Multi-Cams
|
|
|
|
Multi-Cams are virtual Camera Devices that consist of more than one physical Camera Device. For example:
|
|
|
|
- ultra-wide + wide + telephoto = "Triple-Camera"
|
|
- ultra-wide + wide = "Dual-Wide-Camera"
|
|
- wide + telephoto = "Dual-Camera"
|
|
|
|
Benefits of Multi-Cams:
|
|
|
|
- Multi-Cams can smoothly switch between the physical Camera Devices (lenses) while zooming.
|
|
- Multi-Cams can capture Frames from all physical Camera Devices at the same time and fuse them together to create higher-quality Photos.
|
|
|
|
Downsides of Multi-Cams:
|
|
|
|
- The Camera takes longer to initialize and uses more resources
|
|
|
|
To use the "Triple-Camera" in your app, you can just search for a device that contains all three physical Camera Devices:
|
|
|
|
<Tabs
|
|
groupId="component-style"
|
|
defaultValue="hooks"
|
|
values={[
|
|
{label: 'Hooks API', value: 'hooks'},
|
|
{label: 'Imperative API', value: 'imperative'}
|
|
]}>
|
|
<TabItem value="hooks">
|
|
|
|
```ts
|
|
const device = useCameraDevice('back', {
|
|
physicalDevices: [
|
|
'ultra-wide-angle-camera',
|
|
'wide-angle-camera',
|
|
'telephoto-camera'
|
|
]
|
|
})
|
|
```
|
|
|
|
</TabItem>
|
|
<TabItem value="imperative">
|
|
|
|
```ts
|
|
const devices = Camera.getAvailableCameraDevices()
|
|
const device = getCameraDevice(devices, 'back', {
|
|
physicalDevices: [
|
|
'ultra-wide-angle-camera',
|
|
'wide-angle-camera',
|
|
'telephoto-camera'
|
|
]
|
|
})
|
|
```
|
|
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
This will try to find a [`CameraDevice`](/docs/api/interfaces/CameraDevice) that consists of all three physical Camera Devices, or the next best match (e.g. "Dual-Camera", or just a single wide-angle-camera) if not found. With the "Triple-Camera", we can now zoom out to a wider field of view:
|
|
|
|
<div align="center">
|
|
<img class="doc-image" src="/img/multi-camera.gif" />
|
|
</div>
|
|
|
|
If you want to do the filtering/sorting fully yourself, you can also just get all devices, then implement your own filter:
|
|
|
|
<Tabs
|
|
groupId="component-style"
|
|
defaultValue="hooks"
|
|
values={[
|
|
{label: 'Hooks API', value: 'hooks'},
|
|
{label: 'Imperative API', value: 'imperative'}
|
|
]}>
|
|
<TabItem value="hooks">
|
|
|
|
```ts
|
|
const devices = useCameraDevices()
|
|
const device = useMemo(() => findBestDevice(devices), [devices])
|
|
```
|
|
|
|
</TabItem>
|
|
<TabItem value="imperative">
|
|
|
|
```ts
|
|
const devices = Camera.getAvailableCameraDevices()
|
|
const device = findBestDevice(devices)
|
|
```
|
|
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
### Selecting external Cameras
|
|
|
|
VisionCamera supports using `external` Camera Devices, such as:
|
|
- USB Camera Devices (if they support the [USB Video Class (UVC) Specification](https://en.wikipedia.org/wiki/List_of_USB_video_class_devices))
|
|
- [Continuity Camera Devices](https://support.apple.com/en-us/HT213244) (e.g. your iPhone's or Mac's Camera connected through WiFi/Continuity)
|
|
- Bluetooth/WiFi Camera Devices (if they are supported in the platform-native Camera APIs)
|
|
|
|
Since `external` Camera Devices can be plugged in/out at any point, you need to make sure to listen for changes in the Camera Devices list when using `external` Cameras:
|
|
|
|
<Tabs
|
|
groupId="component-style"
|
|
defaultValue="hooks"
|
|
values={[
|
|
{label: 'Hooks API', value: 'hooks'},
|
|
{label: 'Imperative API', value: 'imperative'}
|
|
]}>
|
|
<TabItem value="hooks">
|
|
|
|
The hooks (`useCameraDevice(..)` and `useCameraDevices()`) already automatically listen for Camera Device changes!
|
|
|
|
```ts
|
|
const usbCamera = useCameraDevice('external')
|
|
```
|
|
|
|
</TabItem>
|
|
<TabItem value="imperative">
|
|
|
|
Add a listener by using the [`addCameraDevicesChangedListener(..)`](/docs/api/classes/Camera#addcameradeviceschangedlistener) API:
|
|
|
|
```ts
|
|
const listener = Camera.addCameraDevicesChangedListener((devices) => {
|
|
console.log(`Devices changed: ${devices}`)
|
|
this.usbCamera = devices.find((d) => d.position === "external")
|
|
})
|
|
// ...
|
|
listener.remove()
|
|
```
|
|
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
<br />
|
|
|
|
#### 🚀 Next section: [Camera Lifecycle](lifecycle)
|