Create FRAME_PROCESSORS.md

README.md

@@ -1,14 +1,18 @@
 <table>
 <tr>
 <th>README.md</th>
-<th><a href="./docs/ANIMATED.md">ANIMATED.md</a></th>
 <th><a href="./docs/DEVICES.md">DEVICES.md</a></th>
 <th><a href="./docs/FORMATS.md">FORMATS.md</a></th>
+<th><a href="./docs/FRAME_PROCESSORS.md">FRAME_PROCESSORS.md</a></th>
+<th><a href="./docs/ANIMATED.md">ANIMATED.md</a></th>
 <th><a href="./docs/ERRORS.md">ERRORS.md</a></th>
 </tr>
 </table>
 
 <br/>
+<br/>
+
+<h1 align="center">Camera</h1>
 
 <div align="center">
   <img src="img/11.png" width="55%">

docs/ANIMATED.md

@@ -1,9 +1,10 @@
 <table>
 <tr>
 <th><a href="../README.md">README.md</a></th>
-<th>ANIMATED.md</th>
 <th><a href="./DEVICES.md">DEVICES.md</a></th>
 <th><a href="./FORMATS.md">FORMATS.md</a></th>
+<th><a href="./FRAME_PROCESSORS.md">FRAME_PROCESSORS.md</a></th>
+<th>ANIMATED.md</th>
 <th><a href="./ERRORS.md">ERRORS.md</a></th>
 </tr>
 </table>

docs/DEVICES.md

@@ -1,9 +1,10 @@
 <table>
 <tr>
 <th><a href="../README.md">README.md</a></th>
-<th><a href="./ANIMATED.md">ANIMATED.md</a></th>
 <th>DEVICES.md</th>
 <th><a href="./FORMATS.md">FORMATS.md</a></th>
+<th><a href="./FRAME_PROCESSORS.md">FRAME_PROCESSORS.md</a></th>
+<th><a href="./ANIMATED.md">ANIMATED.md</a></th>
 <th><a href="./ERRORS.md">ERRORS.md</a></th>
 </tr>
 </table>

docs/ERRORS.md

@@ -1,9 +1,10 @@
 <table>
 <tr>
 <th><a href="../README.md">README.md</a></th>
-<th><a href="./ANIMATED.md">ANIMATED.md</a></th>
 <th><a href="./DEVICES.md">DEVICES.md</a></th>
 <th><a href="./FORMATS.md">FORMATS.md</a></th>
+<th><a href="./FRAME_PROCESSORS.md">FRAME_PROCESSORS.md</a></th>
+<th><a href="./ANIMATED.md">ANIMATED.md</a></th>
 <th>ERRORS.md</th>
 </tr>
 </table>

docs/FORMATS.md

@@ -1,9 +1,10 @@
 <table>
 <tr>
 <th><a href="../README.md">README.md</a></th>
-<th><a href="./ANIMATED.md">ANIMATED.md</a></th>
 <th><a href="./DEVICES.md">DEVICES.md</a></th>
 <th>FORMATS.md</th>
+<th><a href="./FRAME_PROCESSORS.md">FRAME_PROCESSORS.md</a></th>
+<th><a href="./ANIMATED.md">ANIMATED.md</a></th>
 <th><a href="./ERRORS.md">ERRORS.md</a></th>
 </tr>
 </table>

docs/FRAME_PROCESSORS.md

@@ -0,0 +1,62 @@
+<table>
+<tr>
+<th><a href="../README.md">README.md</a></th>
+<th><a href="./DEVICES.md">DEVICES.md</a></th>
+<th><a href="./FORMATS.md">FORMATS.md</a></th>
+<th>FRAME_PROCESSORS.md</th>
+<th><a href="./ANIMATED.md">ANIMATED.md</a></th>
+<th><a href="./ERRORS.md">ERRORS.md</a></th>
+</tr>
+</table>
+
+<br/>
+<br/>
+
+<h1 align="center">Frame Processors</h1>
+
+<div>
+<!-- TODO: Demo of QR code scanning or smth -->
+  <img align="right" width="35%" src="../img/ultra-wide-demo.gif">
+</div>
+
+### What are frame processors?
+
+Frame processors are functions that are written in JavaScript (or TypeScript) which can be used to **process frames the camera "sees"**.
+
+For example, you might want to create a QR code scanner _without ever writing native code while still achieving almost-native performance_. Since you can write the scanning part yourself, you can implement a custom QR code system like the one Snapchat uses for Snap-codes.
+
+<div align="center">
+  <img src="../img/snap-code.png" width="15%" />
+</div>
+<br />
+
+Frame processors are by far not limited to QR code detection, other examples include:
+
+* **AI** for **facial recognition**
+* **AI** for **object detection**
+* Using **Tensorflow**, **MLKit Vision** or other libraries (if they provide React Native JSI bindings in the form of "react-native-vision-camera plugins")
+* Creating **realtime video-chats** since you can directly send the camera frames over the network
+* Creating **snapchat-like filters**, e.g. draw a dog-mask filter over the user's face (WIP)
+* Creating **color filters** with depth-detection
+* Using **custom C++ processors** exposed to JS for maximum performance
+
+### Technical
+
+Frame processors are JS functions that will be **workletized** using [react-native-reanimated](https://github.com/software-mansion/react-native-reanimated). They are created on a **separate thread** using a separate Hermes/JSC Runtime and are **invoked synchronously** without ever going over the bridge.
+
+### Example
+
+```tsx
+function App() {
+  const frameProcessor = useFrameProcessor((frame) => {
+    const qrCodes = scanQrCodes(frame)
+    console.log(qrCodes)
+  }, [])
+
+  return (
+    <Camera frameProcessor={frameProcessor} {...cameraProps} />
+  )
+}
+```
+
+> FRAME PROCESSORS ARE STILL WIP