b6a67d5ced
* Clean up Frame Processor * Create FrameProcessorHolder * Create FrameProcessorDelegate in ObjC++ * Move frame processor to FrameProcessorDelegate * Decorate runtime, check for null * Update FrameProcessorDelegate.mm * Cleanup FrameProcessorBindings.mm * Fix RuntimeDecorator.h import * Update FrameProcessorDelegate.mm * "React" -> "React Helper" to avoid confusion * Rename folders again * Fix podspec flattening a lot of headers, causing REA nameclash * Fix header imports to avoid REA naming collision * Lazily initialize jsi::Runtime on DispatchQueue * Install frame processor bindings from Swift * First try to call jsi::Function (frame processor) 👀 * Call viewForReactTag on RCT main thread * Fix bridge accessing * Add more logs * Update CameraViewManager.swift * Add more TODOs * Re-indent .cpp files * Fix RCTTurboModule import podspec * Remove unnecessary include check for swift umbrella header * Merge branch 'main' into frame-processors * Docs: use static width for images (283) * Create validate-cpp.yml * Update a lot of packages to latest * Set SWIFT_VERSION to 5.2 in podspec * Create clean.sh * Delete unused C++ files * podspec: Remove CLANG_CXX_LANGUAGE_STANDARD and OTHER_CFLAGS * Update pod lockfiles * Regenerate lockfiles * Remove IOSLogger * Use NSLog * Create FrameProcessorManager (inherits from REA RuntimeManager) * Create reanimated::RuntimeManager shared_ptr * Re-integrate pods * Add react-native-reanimated >=2 peerDependency * Add metro-config * blacklist -> exclusionList * Try to call worklet * Fix jsi::Value* initializer * Call ShareableValue::adapt (makeShareable) with React/JS Runtime * Add null-checks * Lift runtime manager creation out of delegate, into bindings * Remove debug statement * Make RuntimeManager unique_ptr * Set _FRAME_PROCESSOR * Extract convertJSIFunctionToFrameProcessorCallback * Print frame * Merge branch 'main' into frame-processors * Reformat Swift code * Install reanimated from npm again * Re-integrate Pods * Dependabot: Also scan example/ and docs/ * Update validate-cpp.yml * Create FrameProcessorUtils * Create Frame.h * Abstract HostObject creation away * Fix types * Fix frame processor call * Add todo * Update lockfiles * Add C++ contributing instructions * Update CONTRIBUTING.md * Add android/src/main/cpp to cpplint * Update cpplint.sh * Fix a few cpplint errors * Fix globals * Fix a few more cpplint errors * Update App.tsx * Update AndroidLogger.cpp * Format * Fix cpplint script (check-cpp) * Try to simplify frame processor * y * Update FrameProcessorUtils.mm * Update FrameProcessorBindings.mm * Update CameraView.swift * Update CameraViewManager.m * Restructure everything * fix * Fix `@objc` export (make public) * Refactor installFrameProcessorBindings into FrameProcessorRuntimeManager * Add swift RCTBridge.runOnJS helper * Fix run(onJS) * Add pragma once * Add `&self` to lambda * Update FrameProcessorRuntimeManager.mm * reorder imports * Fix imports * forward declare * Rename extension * Destroy buffer after execution * Add FrameProcessorPluginRegistry base * Merge branch 'main' into frame-processors * Add frameProcessor to types * Update Camera.tsx * Fix rebase merge * Remove movieOutput * Use `useFrameProcessor` * Fix bad merge * Add additional ESLint rules * Update lockfiles * Update CameraViewManager.m * Add support for V8 runtime * Add frame processor plugins API * Print plugin invoke * Fix React Utils in podspec * Fix runOnJS swift name * Remove invalid redecl of `captureSession` * Use REA 2.1.0 which includes all my big PRs 🎉 * Update validate-cpp.yml * Update Podfile.lock * Remove Flipper * Fix dereferencing * Capture `self` by value. Fucking hell, what a dumb mistake. * Override a few HostObject functions * Expose isReady, width, height, bytesPerRow and planesCount * use hook again * Expose property names * FrameProcessor -> Frame * Update CameraView+RecordVideo.swift * Add Swift support for Frame Processors Plugins * Add macros for plugin installation * Add ObjC frame processor plugin * Correctly install frame processor plugins * Don't require custom name for macro * Check if plugin already exists * Implement QR Code Frame Processor Plugin in Swift * Adjust ObjC style frame processor macro * optimize * Add `frameProcessorFrameDropRate` * Fix types * Only log once * Log if it executes slowly * Implement `frameProcessorFps` * Implement manual encoded video recordings * Use recommended video settings * Add fileType types * Ignore if input is not ready for media data * Add completion handler * Add audio buffer sampling * Init only for video frame * use AVAssetWriterInputPixelBufferAdaptor * Remove AVAssetWriterInputPixelBufferAdaptor * Rotate VideoWriter * Always assume portrait orientation * Update RecordingSession.swift * Use a separate Queue for Audio * Format Swift * Update CameraView+RecordVideo.swift * Use `videoQueue` instead of `cameraQueue` * Move example plugins to example app * Fix hardcoded name in plugin macro * QRFrame... -> QRCodeFrame... * Update FrameProcessorPlugin.h * Add example frame processors to JS base * Update QRCodeFrameProcessorPluginSwift.m * Add docs to create FP Plugins * Update FRAME_PROCESSORS_CREATE.mdx * Update FRAME_PROCESSORS_CREATE.mdx * Use `AVAssetWriterInputPixelBufferAdaptor` for efficient pixel buffer recycling * Add customizable `pixelFormat` * Use native format if available * Update project.pbxproj * Set video width and height as source-pixel-buffer attributes * Catch * Update App.tsx * Don't explicitly set video dimensions, let CVPixelBufferPool handle it * Add a few logs * Cleanup * Update CameraView+RecordVideo.swift * Eagerly initialize asset writer to fix stutter at first frame * Use `cameraQueue` DispatchQueue to not block CaptureDataOutputDelegate * Fix duration calculation * cleanup * Cleanup * Swiftformat * Return available video codecs * Only show frame drop notification for video output * Remove photo and video codec functionality It was too much complexity and probably never used anyways. * Revert all android related changes for now * Cleanup * Remove unused header * Update AVAssetWriter.Status+descriptor.swift * Only call Frame Processor for Video Frames * Fix `if` * Add support for Frame Processor plugin parameters/arguments * Fix arg support * Move to JSIUtils.mm * Update JSIUtils.h * Update FRAME_PROCESSORS_CREATE.mdx * Update FRAME_PROCESSORS_CREATE.mdx * Upgrade packages for docs/ * fix docs * Rename * highlight lines * docs * community plugins * Update FRAME_PROCESSOR_CREATE_FINAL.mdx * Update FRAME_PROCESSOR_PLUGIN_LIST.mdx * Update FRAME_PROCESSOR_PLUGIN_LIST.mdx * Update dependencies (1/2) * Update dependencies (2/2) * Update Gemfile.lock * add FP docs * Update README.md * Make `lastFrameProcessor` private * add `frameProcessor` docs * fix docs * adjust docs * Update DEVICES.mdx * fix * s * Add logs demo * add metro restart note * Update FRAME_PROCESSOR_CREATE_PLUGIN_IOS.mdx * Mirror video device * Update AVCaptureVideoDataOutput+mirror.swift * Create .swift-version * Enable whole module optimization * Fix recording mirrored video * Swift format * Clean dictionary on `markInvalid` * Fix cleanup * Add docs for disabling frame processors * Update project.pbxproj * Revert "Update project.pbxproj" This reverts commit e67861e51b88b4888a6940e2d20388f3044211d0. * Log frame drop reason * Format * add more samples * Add clang-format * also check .mm * Revert "also check .mm" This reverts commit 8b9d5e2c29866b05909530d104f6633d6c49eadd. * Revert "Add clang-format" This reverts commit 7643ac808e0fc34567ea1f814e73d84955381636. * Use `kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange` as default * Read matching video attributes from videoSettings * Add TODO * Swiftformat * Conditionally disable frame processors * Assert if trying to use frame processors when disabled * Add frame-processors demo gif * Allow disabling frame processors via `VISION_CAMERA_DISABLE_FRAME_PROCESSORS` * Update FrameProcessorRuntimeManager.mm * Update FRAME_PROCESSORS.mdx * Update project.pbxproj * Update FRAME_PROCESSORS_CREATE_OVERVIEW.mdx
179 lines
6.1 KiB
TypeScript
179 lines
6.1 KiB
TypeScript
import React, { useCallback, useMemo, useState } from 'react';
|
|
import { StyleSheet, View, Image, ActivityIndicator, PermissionsAndroid, Platform } from 'react-native';
|
|
import { Navigation, NavigationFunctionComponent, OptionsModalPresentationStyle } from 'react-native-navigation';
|
|
import Video, { LoadError, OnLoadData } from 'react-native-video';
|
|
import { SAFE_AREA_PADDING } from './Constants';
|
|
import { useIsForeground } from './hooks/useIsForeground';
|
|
import { useIsScreenFocused } from './hooks/useIsScreenFocused';
|
|
import { PressableOpacity } from './views/PressableOpacity';
|
|
import IonIcon from 'react-native-vector-icons/Ionicons';
|
|
import { Alert } from 'react-native';
|
|
import CameraRoll from '@react-native-community/cameraroll';
|
|
import { StatusBarBlurBackground } from './views/StatusBarBlurBackground';
|
|
import type { NativeSyntheticEvent } from 'react-native';
|
|
import type { ImageLoadEventData } from 'react-native';
|
|
|
|
interface MediaProps {
|
|
path: string;
|
|
type: 'video' | 'photo';
|
|
}
|
|
|
|
const requestSavePermission = async (): Promise<boolean> => {
|
|
if (Platform.OS !== 'android') return true;
|
|
|
|
const permission = PermissionsAndroid.PERMISSIONS.WRITE_EXTERNAL_STORAGE;
|
|
if (permission == null) return false;
|
|
let hasPermission = await PermissionsAndroid.check(permission);
|
|
if (!hasPermission) {
|
|
const permissionRequestResult = await PermissionsAndroid.request(permission);
|
|
hasPermission = permissionRequestResult === 'granted';
|
|
}
|
|
return hasPermission;
|
|
};
|
|
|
|
const isVideoOnLoadEvent = (event: OnLoadData | NativeSyntheticEvent<ImageLoadEventData>): event is OnLoadData =>
|
|
'duration' in event && 'naturalSize' in event;
|
|
|
|
export const Media: NavigationFunctionComponent<MediaProps> = ({ componentId, type, path }) => {
|
|
const [hasMediaLoaded, setHasMediaLoaded] = useState(false);
|
|
const isForeground = useIsForeground();
|
|
const isScreenFocused = useIsScreenFocused(componentId);
|
|
const isVideoPaused = !isForeground || !isScreenFocused;
|
|
const [savingState, setSavingState] = useState<'none' | 'saving' | 'saved'>('none');
|
|
|
|
const onClosePressed = useCallback(() => {
|
|
Navigation.dismissModal(componentId);
|
|
}, [componentId]);
|
|
|
|
const onMediaLoad = useCallback((event: OnLoadData | NativeSyntheticEvent<ImageLoadEventData>) => {
|
|
if (isVideoOnLoadEvent(event)) {
|
|
console.log(
|
|
`Video loaded. Size: ${event.naturalSize.width}x${event.naturalSize.height} (${event.naturalSize.orientation}, ${event.duration} seconds)`,
|
|
);
|
|
} else {
|
|
console.log(`Image loaded. Size: ${event.nativeEvent.source.width}x${event.nativeEvent.source.height}`);
|
|
}
|
|
}, []);
|
|
const onMediaLoadEnd = useCallback(() => {
|
|
console.log('media has loaded.');
|
|
setHasMediaLoaded(true);
|
|
}, []);
|
|
const onMediaLoadError = useCallback((error: LoadError) => {
|
|
console.log(`failed to load media: ${JSON.stringify(error)}`);
|
|
}, []);
|
|
|
|
const onSavePressed = useCallback(async () => {
|
|
try {
|
|
setSavingState('saving');
|
|
|
|
const hasPermission = await requestSavePermission();
|
|
if (!hasPermission) {
|
|
Alert.alert('Permission denied!', 'Vision Camera does not have permission to save the media to your camera roll.');
|
|
return;
|
|
}
|
|
await CameraRoll.save(`file://${path}`, {
|
|
type: type,
|
|
});
|
|
setSavingState('saved');
|
|
} catch (e) {
|
|
setSavingState('none');
|
|
Alert.alert('Failed to save!', `An unexpected error occured while trying to save your ${type}. ${e?.message ?? JSON.stringify(e)}`);
|
|
}
|
|
}, [path, type]);
|
|
|
|
const source = useMemo(() => ({ uri: `file://${path}` }), [path]);
|
|
|
|
const screenStyle = useMemo(() => ({ opacity: hasMediaLoaded ? 1 : 0 }), [hasMediaLoaded]);
|
|
|
|
return (
|
|
<View style={[styles.container, screenStyle]}>
|
|
{type === 'photo' && (
|
|
<Image source={source} style={StyleSheet.absoluteFill} resizeMode="cover" onLoadEnd={onMediaLoadEnd} onLoad={onMediaLoad} />
|
|
)}
|
|
{type === 'video' && (
|
|
<Video
|
|
source={source}
|
|
style={StyleSheet.absoluteFill}
|
|
paused={isVideoPaused}
|
|
resizeMode="cover"
|
|
posterResizeMode="cover"
|
|
allowsExternalPlayback={false}
|
|
automaticallyWaitsToMinimizeStalling={false}
|
|
disableFocus={true}
|
|
repeat={true}
|
|
useTextureView={false}
|
|
controls={false}
|
|
playWhenInactive={true}
|
|
ignoreSilentSwitch="ignore"
|
|
onReadyForDisplay={onMediaLoadEnd}
|
|
onLoad={onMediaLoad}
|
|
onError={onMediaLoadError}
|
|
/>
|
|
)}
|
|
|
|
<PressableOpacity style={styles.closeButton} onPress={onClosePressed}>
|
|
<IonIcon name="close" size={35} color="white" style={styles.icon} />
|
|
</PressableOpacity>
|
|
|
|
<PressableOpacity style={styles.saveButton} onPress={onSavePressed} disabled={savingState !== 'none'}>
|
|
{savingState === 'none' && <IonIcon name="download" size={35} color="white" style={styles.icon} />}
|
|
{savingState === 'saved' && <IonIcon name="checkmark" size={35} color="white" style={styles.icon} />}
|
|
{savingState === 'saving' && <ActivityIndicator color="white" />}
|
|
</PressableOpacity>
|
|
|
|
<StatusBarBlurBackground />
|
|
</View>
|
|
);
|
|
};
|
|
|
|
Media.options = {
|
|
modal: {
|
|
swipeToDismiss: false,
|
|
},
|
|
modalPresentationStyle: OptionsModalPresentationStyle.overCurrentContext,
|
|
animations: {
|
|
showModal: {
|
|
waitForRender: true,
|
|
enabled: false,
|
|
},
|
|
dismissModal: {
|
|
enabled: false,
|
|
},
|
|
},
|
|
layout: {
|
|
backgroundColor: 'transparent',
|
|
componentBackgroundColor: 'transparent',
|
|
},
|
|
};
|
|
|
|
const styles = StyleSheet.create({
|
|
container: {
|
|
flex: 1,
|
|
alignItems: 'center',
|
|
justifyContent: 'center',
|
|
backgroundColor: 'white',
|
|
},
|
|
closeButton: {
|
|
position: 'absolute',
|
|
top: SAFE_AREA_PADDING.paddingTop,
|
|
left: SAFE_AREA_PADDING.paddingLeft,
|
|
width: 40,
|
|
height: 40,
|
|
},
|
|
saveButton: {
|
|
position: 'absolute',
|
|
bottom: SAFE_AREA_PADDING.paddingBottom,
|
|
left: SAFE_AREA_PADDING.paddingLeft,
|
|
width: 40,
|
|
height: 40,
|
|
},
|
|
icon: {
|
|
textShadowColor: 'black',
|
|
textShadowOffset: {
|
|
height: 0,
|
|
width: 0,
|
|
},
|
|
textShadowRadius: 1,
|
|
},
|
|
});
|