railbird/react-native-vision-camera
9
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: Update Format.pixelFormat documentation

Browse Source
This commit is contained in:
Marc Rousavy 2021-12-10 09:52:40 +01:00
parent 9301430165
commit 934de142ee
3 changed files with 11 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
docs/docs/guides/FRAME_PROCESSORS.mdx
View File

@ -160,6 +160,13 @@ const frameProcessor = useFrameProcessor((frame) => {
Check out [**Frame Processor community plugins**](/docs/guides/frame-processor-plugin-list) to discover plugins, or [**start creating a plugin yourself**](/docs/guides/frame-processors-plugins-overview)! Check out [**Frame Processor community plugins**](/docs/guides/frame-processor-plugin-list) to discover plugins, or [**start creating a plugin yourself**](/docs/guides/frame-processors-plugins-overview)!
### Selecting a Format for a Frame Processor
When running frame processors, it is often important to choose an appropriate [format](/docs/guides/formats). Here are some general tips to consider:
* If you are running heavy AI/ML calculations in your frame processor, make sure to [select a format](/docs/guides/formats) that has a lower resolution to optimize it's performance.
* Sometimes a frame processor plugin only works with specific [pixel formats](/docs/api/interfaces/CameraDeviceFormat#pixelformat). Some plugins (like MLKit) don't work with `x420`.
### Benchmarks ### Benchmarks
Frame Processors are _really_ fast. I have used [MLKit Vision Image Labeling](https://firebase.google.com/docs/ml-kit/ios/label-images) to label 4k Camera frames in realtime, and measured the following results: Frame Processors are _really_ fast. I have used [MLKit Vision Image Labeling](https://firebase.google.com/docs/ml-kit/ios/label-images) to label 4k Camera frames in realtime, and measured the following results:

2
ios/CameraView+RecordVideo.swift
View File

@ -110,7 +110,7 @@ extension CameraView: AVCaptureVideoDataOutputSampleBufferDelegate, AVCaptureAud
callback.reject(error: .capture(.createRecorderError(message: "Failed to get video settings!"))) callback.reject(error: .capture(.createRecorderError(message: "Failed to get video settings!")))
return return
} }
// get pixel format (420f, 420v) // get pixel format (420f, 420v, x420)
let pixelFormat = CMFormatDescriptionGetMediaSubType(videoInput.device.activeFormat.formatDescription) let pixelFormat = CMFormatDescriptionGetMediaSubType(videoInput.device.activeFormat.formatDescription)
self.recordingSession!.initializeVideoWriter(withSettings: videoSettings, self.recordingSession!.initializeVideoWriter(withSettings: videoSettings,
pixelFormat: pixelFormat) pixelFormat: pixelFormat)

4
src/CameraDevice.ts
View File

@ -173,7 +173,9 @@ export interface CameraDeviceFormat {
*/ */
videoStabilizationModes: VideoStabilizationMode[]; videoStabilizationModes: VideoStabilizationMode[];
/** /**
* Specifies the pixel format on iOS. e.g. 420v, 420f * Specifies this format's pixel format. The pixel format specifies how the individual pixels are interpreted as a visual image.
*
* The most common format is `420v`. Some formats (like `x420`) are not compatible with some frame processor plugins (e.g. MLKit)
*/ */
pixelFormat: PixelFormat; pixelFormat: PixelFormat;
} }