150 lines
6.1 KiB
Markdown
150 lines
6.1 KiB
Markdown
<table>
|
|
<tr>
|
|
<th><a href="../README.md">README</a></th>
|
|
<th><a href="./SETUP.md">SETUP</a></th>
|
|
<th><a href="./DEVICES.md">DEVICES</a></th>
|
|
<th>FORMATS</th>
|
|
<th><a href="./FRAME_PROCESSORS.md">FRAME_PROCESSORS</a></th>
|
|
<th><a href="./ANIMATED.md">ANIMATED</a></th>
|
|
<th><a href="./ERRORS.md">ERRORS</a></th>
|
|
</tr>
|
|
</table>
|
|
|
|
<br/>
|
|
|
|
<h1 align="center">Formats</h1>
|
|
|
|
<div>
|
|
<img align="right" width="35%" src="../img/example.png">
|
|
</div>
|
|
|
|
### What are camera formats?
|
|
|
|
Each camera device (see [DEVICES.md](./DEVICES.md)) provides a number of capture formats that have different specifications. There are formats specifically designed for high-resolution photo capture, which have very high photo output quality but in return only support frame-rates of up to 30 FPS. On the other side, there might be formats that are designed for slow-motion video capture which have frame-rates up to 240 FPS.
|
|
|
|
### What if I don't want to choose a format?
|
|
|
|
If you don't want to specify the best format for your camera device, you don't have to. The Camera _automatically chooses the best matching format_ for the current camera device. This is why the Camera's `format` property is _optional_.
|
|
|
|
If you don't want to do a lot of filtering, but still want to let the camera know what your intentions are, you can use the Camera's `preset` property. (🔗 See the [CameraPreset.ts](../src/CameraPreset.ts) type for more information about presets)
|
|
|
|
For example, use the `'medium'` preset if you want to create a video-chat application that shouldn't excessively use mobile data:
|
|
|
|
```tsx
|
|
function App() {
|
|
const devices = useCameraDevices()
|
|
const device = devices.back
|
|
|
|
return (
|
|
<Camera
|
|
style={StyleSheet.absoluteFill}
|
|
device={device}
|
|
preset="medium"
|
|
/>
|
|
)
|
|
}
|
|
```
|
|
|
|
### What you need to know about cameras
|
|
|
|
To understand a bit more about camera formats, you first need to understand a few "general camera basics":
|
|
|
|
* Each camera device is built differently, e.g. _Telephoto devices_ often don't provide frame-rates as high as _Wide-Angle devices_.
|
|
* Formats are designed for specific use-cases, so formats with high resolution photo output don't support frame-rates as high as formats with lower resolution.
|
|
* Different formats provide different field-of-views (FOV), maximum zoom factors, color spaces (iOS only), resolutions, frame rate ranges, and systems to assist with capture (auto-focus systems, video stabilization systems, ...)
|
|
|
|
### Get started
|
|
|
|
Each application has different needs, so the format filtering is up to you.
|
|
|
|
To get all available formats, simply use the `CameraDevice`'s `.formats` property. See how to get a camera device in the [DEVICES.md](./DEVICES.md) doc.
|
|
|
|
> Note: You can also manually get all camera devices and decide which device to use based on the available `formats`. In fact, this is how we do it in the [Cuvent](https://cuvent.com) app.
|
|
|
|
This example shows how you would pick the format with the _highest frame rate_:
|
|
|
|
```tsx
|
|
function getMaxFps(format: CameraDeviceFormat): number {
|
|
return format.frameRateRanges.reduce((prev, curr) => {
|
|
if (curr.maxFrameRate > prev) return curr.maxFrameRate;
|
|
else return prev;
|
|
}, 0);
|
|
}
|
|
|
|
function App() {
|
|
const devices = useCameraDevices('wide-angle-camera')
|
|
const device = devices.back
|
|
|
|
const format = useMemo(() => {
|
|
return device?.formats.reduce((prev, curr) => {
|
|
if (prev == null) return curr;
|
|
if (getMaxFps(curr) > getMaxFps(prev)) return curr;
|
|
else return prev;
|
|
}, undefined);
|
|
}, [device?.formats]);
|
|
|
|
return (
|
|
<Camera
|
|
style={StyleSheet.absoluteFill}
|
|
device={device}
|
|
format={format}
|
|
/>
|
|
)
|
|
}
|
|
```
|
|
|
|
Note that you don't want to simply pick the highest frame rate, as those formats often have incredibly low resolutions. You want to find a balance between high frame rate and high resolution, so instead you might want to use the `.sort` function.
|
|
|
|
### Sort
|
|
|
|
To sort your formats, create a custom comparator function which will be used as the `.sort` function's argument. The custom comparator then compares formats, preferring ones with higher frame rate AND higher resolution.
|
|
|
|
Implement this however you want, I personally use a "point-based system":
|
|
|
|
```ts
|
|
export const sortFormatsByResolution = (left: CameraDeviceFormat, right: CameraDeviceFormat): number => {
|
|
// in this case, points aren't "normalized" (e.g. higher resolution = 1 point, lower resolution = -1 points)
|
|
let leftPoints = left.photoHeight * left.photoWidth;
|
|
let rightPoints = right.photoHeight * right.photoWidth;
|
|
|
|
if (left.videoHeight != null && left.videoWidth != null && right.videoHeight != null && right.videoWidth != null) {
|
|
leftPoints += left.videoWidth * left.videoHeight;
|
|
rightPoints += right.videoWidth * right.videoHeight;
|
|
}
|
|
|
|
// you can also add points for FPS, etc
|
|
|
|
return rightPoints - leftPoints;
|
|
};
|
|
|
|
// and then call it:
|
|
const formats = useMemo(() => device?.formats.sort(sortFormatsByResolution), [device?.formats])
|
|
```
|
|
|
|
Be careful that you don't `filter` out a lot of formats since you might end up having no format to use at all. (_Remember; not all devices support e.g. 240 FPS._) Always sort them and pick the best format, that way you are guaranteed to have a format available, even if your desired specifications aren't fully met.
|
|
|
|
### Props
|
|
|
|
The `Camera` View provides a few props that depend on the specified `format`. For example, you can only set the `fps` prop to a value that is supported by the current `format`. So if you have a format that supports 240 FPS, you can set the `fps` to `240`:
|
|
|
|
```tsx
|
|
function App() {
|
|
return (
|
|
<Camera
|
|
style={StyleSheet.absoluteFill}
|
|
device={device}
|
|
format={format}
|
|
fps={240}
|
|
/>
|
|
)
|
|
}
|
|
```
|
|
|
|
> Note: You should always verify that the format supports the desired FPS, and fall back to `undefined` (or a value that is supported, like `30`) if it doesn't.
|
|
|
|
Other props that depend on the `format`:
|
|
|
|
* `hdr`: Enables HDR photo or video capture and preview
|
|
* `lowLightBoost`: Enables a night-mode/low-light-boost for photo or video capture and preview
|
|
* `colorSpace`: Uses the specified color-space for photo or video capture and preview (iOS only since Android only uses `YUV`)
|