react-native-vision-camera/docs/FORMATS.md

README	SETUP	DEVICES	FORMATS	FRAME_PROCESSORS	ANIMATED	ERRORS

Formats

What are camera formats?

Each camera device (see DEVICES.md) provides a number of capture formats that have different specifications. There are formats specifically designed for high-resolution photo capture, which have very high photo output quality but in return only support frame-rates of up to 30 FPS. On the other side, there might be formats that are designed for slow-motion video capture which have frame-rates up to 240 FPS.

What if I don't want to choose a format?

If you don't want to specify the best format for your camera device, you don't have to. The Camera automatically chooses the best matching format for the current camera device. This is why the Camera's format property is optional.

If you don't want to do a lot of filtering, but still want to let the camera know what your intentions are, you can use the Camera's preset property. (🔗 See the CameraPreset.ts type for more information about presets)

For example, use the 'medium' preset if you want to create a video-chat application that shouldn't excessively use mobile data:

function App() {
  const devices = useCameraDevices()
  const device = devices.back

  return (
    <Camera
      style={StyleSheet.absoluteFill}
      device={device}
      preset="medium"
    />
  )
}

What you need to know about cameras

To understand a bit more about camera formats, you first need to understand a few "general camera basics":

• Each camera device is built differently, e.g. Telephoto devices often don't provide frame-rates as high as Wide-Angle devices.
• Formats are designed for specific use-cases, so formats with high resolution photo output don't support frame-rates as high as formats with lower resolution.
• Different formats provide different field-of-views (FOV), maximum zoom factors, color spaces (iOS only), resolutions, frame rate ranges, and systems to assist with capture (auto-focus systems, video stabilization systems, ...)

Get started

Each application has different needs, so the format filtering is up to you.

To get all available formats, simply use the CameraDevice's .formats property. See how to get a camera device in the DEVICES.md doc.

Note: You can also manually get all camera devices and decide which device to use based on the available formats. In fact, this is how we do it in the Cuvent app.

This example shows how you would pick the format with the highest frame rate:

function getMaxFps(format: CameraDeviceFormat): number {
  return format.frameRateRanges.reduce((prev, curr) => {
    if (curr.maxFrameRate > prev) return curr.maxFrameRate;
    else return prev;
  }, 0);
}

function App() {
  const devices = useCameraDevices('wide-angle-camera')
  const device = devices.back

  const format = useMemo(() => {
    return device?.formats.reduce((prev, curr) => {
      if (prev == null) return curr;
      if (getMaxFps(curr) > getMaxFps(prev)) return curr;
      else return prev;
    }, undefined);
  }, [device?.formats]);

  return (
    <Camera
      style={StyleSheet.absoluteFill}
      device={device}
      format={format}
    />
  )
}

Note that you don't want to simply pick the highest frame rate, as those formats often have incredibly low resolutions. You want to find a balance between high frame rate and high resolution, so instead you might want to use the .sort function.

Sort

To sort your formats, create a custom comparator function which will be used as the .sort function's argument. The custom comparator then compares formats, preferring ones with higher frame rate AND higher resolution.

Implement this however you want, I personally use a "point-based system":

export const sortFormatsByResolution = (left: CameraDeviceFormat, right: CameraDeviceFormat): number => {
  // in this case, points aren't "normalized" (e.g. higher resolution = 1 point, lower resolution = -1 points)
  let leftPoints = left.photoHeight * left.photoWidth;
  let rightPoints = right.photoHeight * right.photoWidth;

  if (left.videoHeight != null && left.videoWidth != null && right.videoHeight != null && right.videoWidth != null) {
    leftPoints += left.videoWidth * left.videoHeight;
    rightPoints += right.videoWidth * right.videoHeight;
  }

  // you can also add points for FPS, etc

  return rightPoints - leftPoints;
};

// and then call it:
const formats = useMemo(() => device?.formats.sort(sortFormatsByResolution), [device?.formats])

Be careful that you don't filter out a lot of formats since you might end up having no format to use at all. (Remember; not all devices support e.g. 240 FPS.) Always sort them and pick the best format, that way you are guaranteed to have a format available, even if your desired specifications aren't fully met.

Props

The Camera View provides a few props that depend on the specified format. For example, you can only set the fps prop to a value that is supported by the current format. So if you have a format that supports 240 FPS, you can set the fps to 240:

function App() {
  return (
    <Camera
      style={StyleSheet.absoluteFill}
      device={device}
      format={format}
      fps={240}
    />
  )
}

Note: You should always verify that the format supports the desired FPS, and fall back to undefined (or a value that is supported, like 30) if it doesn't.

Other props that depend on the format:

• hdr: Enables HDR photo or video capture and preview
• lowLightBoost: Enables a night-mode/low-light-boost for photo or video capture and preview
• colorSpace: Uses the specified color-space for photo or video capture and preview (iOS only since Android only uses YUV)

🚀 Next section: FRAME_PROCESSORS