236 lines
		
	
	
		
			9.5 KiB
		
	
	
	
		
			Plaintext
		
	
	
	
	
	
			
		
		
	
	
			236 lines
		
	
	
		
			9.5 KiB
		
	
	
	
		
			Plaintext
		
	
	
	
	
	
| ---
 | |
| id: devices
 | |
| title: Camera Devices
 | |
| sidebar_label: Camera Devices
 | |
| ---
 | |
| 
 | |
| import Tabs from '@theme/Tabs'
 | |
| import TabItem from '@theme/TabItem'
 | |
| import useBaseUrl from '@docusaurus/useBaseUrl'
 | |
| 
 | |
| <div class="image-container">
 | |
|   <svg xmlns="http://www.w3.org/2000/svg" width="283" height="535">
 | |
|     <image href={useBaseUrl("img/demo.gif")} x="18" y="33" width="247" height="469" />
 | |
|     <image href={useBaseUrl("img/frame.png")} width="283" height="535" />
 | |
|   </svg>
 | |
| </div>
 | |
| 
 | |
| ## What are Camera Devices?
 | |
| 
 | |
| Camera Devices are the physical (or "virtual") devices that can be used to record videos or capture photos.
 | |
| 
 | |
| * **Physical**: A physical Camera Device is a **camera lens on your phone**. Different physical Camera Devices have different specifications, such as different capture formats, resolutions, zoom levels, and more. Some phones have multiple physical Camera Devices.
 | |
| 
 | |
|   > Examples: _"Backside Wide-Angle Camera"_, _"Frontside Wide-Angle Camera (FaceTime HD)"_, _"Ultra-Wide-Angle back camera"_
 | |
| * **Virtual**: A virtual camera device is a **combination of one or more physical camera devices**, and provides features such as _virtual-device-switchover_ while zooming (see video on the right) or _combined photo delivery_ from all physical cameras to produce higher quality images.
 | |
| 
 | |
|   > Examples: _"Triple-Camera"_, _"Dual-Wide-Angle Camera"_
 | |
| 
 | |
| ## Select the default Camera
 | |
| 
 | |
| If you simply want to use the default [`CameraDevice`](/docs/api/interfaces/CameraDevice), you can just use whatever is available:
 | |
| 
 | |
| <Tabs
 | |
|   groupId="component-style"
 | |
|   defaultValue="hooks"
 | |
|   values={[
 | |
|     {label: 'Hooks API', value: 'hooks'},
 | |
|     {label: 'Imperative API', value: 'imperative'}
 | |
|   ]}>
 | |
| <TabItem value="hooks">
 | |
| 
 | |
| ```ts
 | |
| const device = useCameraDevice('back')
 | |
| ```
 | |
| 
 | |
| </TabItem>
 | |
| <TabItem value="imperative">
 | |
| 
 | |
| ```ts
 | |
| const devices = Camera.getAvailableCameraDevices()
 | |
| const device = devices.find((d) => d.position === 'back')
 | |
| ```
 | |
| 
 | |
| </TabItem>
 | |
| </Tabs>
 | |
| 
 | |
| And VisionCamera will automatically find the best matching [`CameraDevice`](/docs/api/interfaces/CameraDevice) for you!
 | |
| 
 | |
| **🚀 Continue with: [Camera Lifecycle](lifecycle)**
 | |
| 
 | |
| ## Custom Device Selection
 | |
| 
 | |
| For advanced use-cases, you might want to select a different [`CameraDevice`](/docs/api/interfaces/CameraDevice) for your app.
 | |
| 
 | |
| A [`CameraDevice`](/docs/api/interfaces/CameraDevice) consists of the following specifications:
 | |
| 
 | |
| - [`id`](/docs/api/interfaces/CameraDevice#id): A unique ID used to identify this Camera Device
 | |
| - [`position`](/docs/api/interfaces/CameraDevice#position): The position of this Camera Device relative to the phone
 | |
|   - `back`: The Camera Device is located on the back of the phone
 | |
|   - `front`: The Camera Device is located on the front of the phone
 | |
|   - `external`: The Camera Device is an external device. These devices can be either:
 | |
|     - USB Camera Devices (if they support the [USB Video Class (UVC) Specification](https://en.wikipedia.org/wiki/List_of_USB_video_class_devices))
 | |
|     - [Continuity Camera Devices](https://support.apple.com/en-us/HT213244) (e.g. your iPhone's or Mac's Camera connected through WiFi/Continuity)
 | |
|     - Bluetooth/WiFi Camera Devices (if they are supported in the platform-native Camera APIs)
 | |
| - [`physicalDevices`](/docs/api/interfaces/CameraDevice#physicaldevices): The physical Camera Devices (lenses) this Camera Device consists of. This can either be one of these values ("physical" device) or any combination of these values ("virtual" device):
 | |
|   - `ultra-wide-angle-camera`: The "fish-eye" camera for 0.5x zoom
 | |
|   - `wide-angle-camera`: The "default" camera for 1x zoom
 | |
|   - `telephoto-camera`: A zoomed-in camera for 3x zoom
 | |
| - [`sensorOrientation`](/docs/api/interfaces/CameraDevice#sensororientation): The orientation of the Camera sensor/lens relative to the phone. Cameras are usually in `landscapeLeft` orientation, meaning they are rotated by 90°. This includes their resolutions, so a 4k format might be 3840x2160, not 2160x3840
 | |
| - [`minZoom`](/docs/api/interfaces/CameraDevice#minzoom): The minimum possible zoom factor for this Camera Device. If this is a multi-cam, this is the point where the device with the widest field of view is used (e.g. ultra-wide)
 | |
| - [`maxZoom`](/docs/api/interfaces/CameraDevice#maxzoom): The maximum possible zoom factor for this Camera Device. If this is a multi-cam, this is the point where the device with the lowest field of view is used (e.g. telephoto)
 | |
| - [`neutralZoom`](/docs/api/interfaces/CameraDevice#neutralzoom): A value between `minZoom` and `maxZoom` where the "default" Camera Device is used (e.g. wide-angle). When using multi-cams, make sure to start off at this zoom level, so the user can optionally zoom out to the ultra-wide-angle Camera instead of already starting zoomed out
 | |
| - [`formats`](/docs/api/interfaces/CameraDevice#formats): The list of [`CameraDeviceFormat`s](/docs/api/interfaces/CameraDeviceFormat) (See ["Camera Formats"](/docs/guides/formats)) this Camera Device supports. A format specifies:
 | |
|   - Video Resolution (see ["Formats: Video Resolution"](/docs/guides/formats#video-resolution))
 | |
|   - Photo Resolution (see ["Formats: Photo Resolution"](/docs/guides/formats#photo-resolution))
 | |
|   - FPS (see ["Formats: FPS"](/docs/guides/formats#fps))
 | |
|   - Video Stabilization Mode (see: ["Formats: Video Stabilization Mode"](/docs/guides/formats#videostabilization))
 | |
|   - Pixel Format (see: ["Formats: Pixel Format"](/docs/guides/formats#pixelformat))
 | |
| 
 | |
| ### Examples on an iPhone
 | |
| 
 | |
| Here's a list of some Camera Devices an iPhone 13 Pro has:
 | |
| 
 | |
| - Back Wide Angle Camera (`['wide-angle-camera']`)
 | |
| - Back Ultra-Wide Angle Camera (`['ultra-wide-angle-camera']`)
 | |
| - Back Telephoto Camera (`['telephoto-camera']`)
 | |
| - Back Dual Camera (Wide + Telephoto)
 | |
| - Back Dual-Wide Camera (Ultra-Wide + Wide)
 | |
| - Back Triple Camera (Ultra-Wide + Wide + Telephoto)
 | |
| - Back LiDAR Camera (Wide + LiDAR-Depth)
 | |
| - Front Wide Angle (`['wide-angle-camera']`)
 | |
| - Front True-Depth (Wide + Depth)
 | |
| 
 | |
| ### Selecting Multi-Cams
 | |
| 
 | |
| Multi-Cams are virtual Camera Devices that consist of more than one physical Camera Device. For example:
 | |
| 
 | |
| - ultra-wide + wide + telephoto = "Triple-Camera"
 | |
| - ultra-wide + wide = "Dual-Wide-Camera"
 | |
| - wide + telephoto = "Dual-Camera"
 | |
| 
 | |
| Benefits of Multi-Cams:
 | |
| 
 | |
| - Multi-Cams can smoothly switch between the physical Camera Devices (lenses) while zooming.
 | |
| - Multi-Cams can capture Frames from all physical Camera Devices at the same time and fuse them together to create higher-quality Photos.
 | |
| 
 | |
| Downsides of Multi-Cams:
 | |
| 
 | |
| - The Camera takes longer to initialize and uses more resources
 | |
| 
 | |
| To use the "Triple-Camera" in your app, you can just search for a device that contains all three physical Camera Devices:
 | |
| 
 | |
| <Tabs
 | |
|   groupId="component-style"
 | |
|   defaultValue="hooks"
 | |
|   values={[
 | |
|     {label: 'Hooks API', value: 'hooks'},
 | |
|     {label: 'Imperative API', value: 'imperative'}
 | |
|   ]}>
 | |
| <TabItem value="hooks">
 | |
| 
 | |
| ```ts
 | |
| const device = useCameraDevice('back', {
 | |
|   physicalDevices: [
 | |
|     'ultra-wide-angle-camera',
 | |
|     'wide-angle-camera',
 | |
|     'telephoto-camera'
 | |
|   ]
 | |
| })
 | |
| ```
 | |
| 
 | |
| </TabItem>
 | |
| <TabItem value="imperative">
 | |
| 
 | |
| ```ts
 | |
| const devices = Camera.getAvailableCameraDevices()
 | |
| const device = getCameraDevice(devices, 'back', {
 | |
|   physicalDevices: [
 | |
|     'ultra-wide-angle-camera',
 | |
|     'wide-angle-camera',
 | |
|     'telephoto-camera'
 | |
|   ]
 | |
| })
 | |
| ```
 | |
| 
 | |
| </TabItem>
 | |
| </Tabs>
 | |
| 
 | |
| This will try to find a [`CameraDevice`](/docs/api/interfaces/CameraDevice) that consists of all three physical Camera Devices, or the next best match (e.g. "Dual-Camera", or just a single wide-angle-camera) if not found. With the "Triple-Camera", we can now zoom out to a wider field of view:
 | |
| 
 | |
| <div align="center">
 | |
|   <img class="doc-image" src="/img/multi-camera.gif" />
 | |
| </div>
 | |
| 
 | |
| If you want to do the filtering/sorting fully yourself, you can also just get all devices, then implement your own filter:
 | |
| 
 | |
| <Tabs
 | |
|   groupId="component-style"
 | |
|   defaultValue="hooks"
 | |
|   values={[
 | |
|     {label: 'Hooks API', value: 'hooks'},
 | |
|     {label: 'Imperative API', value: 'imperative'}
 | |
|   ]}>
 | |
| <TabItem value="hooks">
 | |
| 
 | |
| ```ts
 | |
| const devices = useCameraDevices()
 | |
| const device = useMemo(() => findBestDevice(devices), [devices])
 | |
| ```
 | |
| 
 | |
| </TabItem>
 | |
| <TabItem value="imperative">
 | |
| 
 | |
| ```ts
 | |
| const devices = Camera.getAvailableCameraDevices()
 | |
| const device = findBestDevice(devices)
 | |
| ```
 | |
| 
 | |
| </TabItem>
 | |
| </Tabs>
 | |
| 
 | |
| ### Selecting external Cameras
 | |
| 
 | |
| VisionCamera supports using `external` Camera Devices, such as:
 | |
| - USB Camera Devices (if they support the [USB Video Class (UVC) Specification](https://en.wikipedia.org/wiki/List_of_USB_video_class_devices))
 | |
| - [Continuity Camera Devices](https://support.apple.com/en-us/HT213244) (e.g. your iPhone's or Mac's Camera connected through WiFi/Continuity)
 | |
| - Bluetooth/WiFi Camera Devices (if they are supported in the platform-native Camera APIs)
 | |
| 
 | |
| Since `external` Camera Devices can be plugged in/out at any point, you need to make sure to listen for changes in the Camera Devices list when using `external` Cameras:
 | |
| 
 | |
| <Tabs
 | |
|   groupId="component-style"
 | |
|   defaultValue="hooks"
 | |
|   values={[
 | |
|     {label: 'Hooks API', value: 'hooks'},
 | |
|     {label: 'Imperative API', value: 'imperative'}
 | |
|   ]}>
 | |
| <TabItem value="hooks">
 | |
| 
 | |
| The hooks (`useCameraDevice(..)` and `useCameraDevices()`) already automatically listen for Camera Device changes!
 | |
| 
 | |
| ```ts
 | |
| const usbCamera = useCameraDevice('external')
 | |
| ```
 | |
| 
 | |
| </TabItem>
 | |
| <TabItem value="imperative">
 | |
| 
 | |
| Add a listener by using the [`addCameraDevicesChangedListener(..)`](/docs/api/classes/Camera#addcameradeviceschangedlistener) API:
 | |
| 
 | |
| ```ts
 | |
| const listener = Camera.addCameraDevicesChangedListener((devices) => {
 | |
|   console.log(`Devices changed: ${devices}`)
 | |
|   this.usbCamera = devices.find((d) => d.position === "external")
 | |
| })
 | |
| // ...
 | |
| listener.remove()
 | |
| ```
 | |
| 
 | |
| </TabItem>
 | |
| </Tabs>
 | |
| 
 | |
| <br />
 | |
| 
 | |
| #### 🚀 Next section: [Camera Lifecycle](lifecycle)
 |