🐛 [V4] Android GPS location cache issue

[idrisssakhi]

What's happening?

The GPS coordinates are fetched from cache on Android, so we need to disable the cache when streaming location on Android. (Screenshot included, the GPSTimeStamp 07:53, image was taken at 11:20) The heading value (GPSImageDirection), are not available on android.

Reproduceable Code

import { ReactElement, useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { ImageRequireSource, Pressable, StyleSheet, View, ViewStyle } from 'react-native';

import { faCameraRotate } from '@fortawesome/free-solid-svg-icons/faCameraRotate';
import { faMoon } from '@fortawesome/free-solid-svg-icons/faMoon';
import { FontAwesomeIcon } from '@fortawesome/react-native-fontawesome';
import { writeAsync } from '@lodev09/react-native-exify';
import { GeolocationResponse } from '@react-native-community/geolocation';
import CompassHeading from 'react-native-compass-heading';
import Reanimated, {
  interpolate,
  useAnimatedProps,
  useAnimatedRef,
  useAnimatedStyle,
  useSharedValue,
  withSpring,
  withTiming,
} from 'react-native-reanimated';

import { useSafeAreaInsets } from 'react-native-safe-area-context';
import {
  Camera,
  CameraPosition,
  CameraProps,
  PhotoFile,
  TakePhotoOptions,
  useCameraDevice,
  useCameraDevices,
  useCameraFormat,
} from 'react-native-vision-camera';

import HDRLogo from 'assets/icons/camera-hdr/hdr.svg';

import { filePrefix, isAndroid } from 'src/Common/Constants';
import { usePrevious } from 'src/Common/Hooks';
import { Color, style as styleUtils } from 'src/Common/Types';

import { IconButton } from '../Buttons';
import { FullScreenLoader } from '../Loaders';

export interface CameraViewProps {
  onTakePhoto: (photo: PhotoFile, photoPosition?: GeolocationResponse) => void;
  isActive?: boolean;
  allowFrontCamera?: boolean;
  enableZoomGesture?: boolean;
}

const flashIcon = new Map<TakePhotoOptions['flash'], ImageRequireSource>([
  ['auto', require('assets/icons/camera-flash-auto-icon/camera-flash-auto-icon.png')],
  ['on', require('assets/icons/camera-flash-on-icon/camera-flash-on-icon.png')],
  ['off', require('assets/icons/camera-flash-off-icon/camera-flash-off-icon.png')],
]);

/**
 * Returns next value for flash mode based on current, passed as argument
 *
 * @param flashMode - Current flash mode
 */
const getNextFlashMode = (flashMode: TakePhotoOptions['flash']): TakePhotoOptions['flash'] => {
  const flashModesOrder = Array.from(flashIcon.keys());
  const currentIndex = flashModesOrder.findIndex((mode) => mode === flashMode);
  if (currentIndex > -1) {
    return flashModesOrder[currentIndex >= flashModesOrder.length - 1 ? 0 : currentIndex + 1];
  }
  return flashModesOrder[0];
};

/**
 * Returns icon of flash mode
 *
 * @param flashMode - Flash mode to get icon for
 */
const getFlashModeIcon = (flashMode: TakePhotoOptions['flash']): ImageRequireSource => {
  return flashIcon.get(flashMode) ?? (flashIcon.get('auto') as ImageRequireSource);
};

const ReanimatedCamera = Reanimated.createAnimatedComponent<CameraProps>(Camera);
Reanimated.addWhitelistedNativeProps({
  zoom: true,
});

export const CameraView = ({
  onTakePhoto,
  isActive = true,
  allowFrontCamera = false,
  enableZoomGesture = true,
}: CameraViewProps): ReactElement => {
  const { bottom: bottomInset, top: topInset } = useSafeAreaInsets();
  const devices = useCameraDevices();

  const [cameraPosition, setCameraPosition] = useState<CameraPosition>('back');
  const [flash, setFlash] = useState<TakePhotoOptions['flash']>('auto');
  const [actualPhoto, setActualPhoto] = useState<PhotoFile>();
  const [isHdrActive, setIsHdrActive] = useState(false);
  const [isLowLightActive, setIsLowLightActive] = useState(false);

  const cameraRef = useAnimatedRef<Camera>();
  const device = useCameraDevice(cameraPosition, {
    physicalDevices: ['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera'],
  });
  const format = useCameraFormat(device, [{ photoResolution: 'max' }]);

  const fillScale = useSharedValue(0);

  // https://mrousavy.com/react-native-vision-camera/docs/guides/zooming
  const neutralZoom = useMemo(() => device?.neutralZoom ?? 0, [device?.neutralZoom]);

  const cameraZoom = useSharedValue(neutralZoom);

  const actualHeading = useRef<number>(0);

  // reset zoom value on change camera position
  const prevCameraPosition = usePrevious(cameraPosition);
  const resetCameraZoom = useCallback(() => {
    cameraZoom.value = withSpring(neutralZoom);
  }, [cameraZoom, neutralZoom]);

  useEffect(() => {
    if (prevCameraPosition !== cameraPosition) {
      resetCameraZoom();
    }
  }, [prevCameraPosition, cameraPosition, device, resetCameraZoom]);

  const animatedProps = useAnimatedProps<CameraProps>(() => ({ zoom: cameraZoom.value }), [cameraZoom]);

  const buttonFillAnimationStyle: ViewStyle = useAnimatedStyle(() => {
    'worklet';
    return {
      width: interpolate(fillScale.value, [0, 1], [0, 80]),
      height: interpolate(fillScale.value, [0, 1], [0, 80]),
      borderRadius: interpolate(fillScale.value, [0, 1], [0, 40]),
    };
  }, [fillScale.value]);

  const onFlashButtonPress = useCallback(() => {
    setFlash(getNextFlashMode(flash));
  }, [flash]);

  const handleHdrSwitch = useCallback(() => {
    setIsHdrActive((prev) => !prev);
  }, []);

  const handleLowLightBoost = useCallback(() => {
    setIsLowLightActive((prev) => !prev);
  }, []);

  useEffect(() => {
    if (isAndroid) {
      const degree_update_rate = 3;
      CompassHeading.start(degree_update_rate, ({ heading }: { heading: number }) => {
        actualHeading.current = heading;
      });
    }

    return () => {
      isAndroid && CompassHeading.stop();
    };
  }, []);

  const onCameraPhotoButtonPress = useCallback(async () => {
    try {
      fillScale.value = withTiming(1, { duration: isAndroid ? 750 : 125 });

      const cameraPhoto = await cameraRef.current?.takePhoto({ flash });
      if (cameraPhoto) {
        const photoPath = cameraPhoto.path?.startsWith(filePrefix)
          ? cameraPhoto.path
          : `${filePrefix}${cameraPhoto.path}`;

        setActualPhoto({
          ...cameraPhoto,
          path: photoPath,
        });

        // proto preview is opened, reset zoom
        enableZoomGesture && resetCameraZoom();
      }
    } catch (e) {
      console.warn(e);
    } finally {
      fillScale.value = 0;
    }
  }, [fillScale, cameraRef, flash, enableZoomGesture, resetCameraZoom]);

  useEffect(() => {
    (async () => {
      if (actualPhoto) {
        try {
          if (isAndroid) {
            const result = await writeAsync(actualPhoto.path, {
              GPSImgDirection: actualHeading.current,
              GPSImgDirectionRef: actualHeading.current.toString(),
            });
            onTakePhoto({ ...actualPhoto, path: result?.uri ?? actualPhoto.path });
          } else {
            onTakePhoto(actualPhoto);
          }
        } catch (e) {
          console.warn('[error] - while writing exif data', e);
        } finally {
          setActualPhoto(undefined);
        }
      }
    })();
  }, [actualHeading, actualPhoto, onTakePhoto]);

  const canChangeCameraPosition = useMemo(
    () =>
      allowFrontCamera &&
      (['front', 'back'] as CameraPosition[]).every(
        (position) => devices.findIndex((dev) => dev.position === position) !== -1,
      ),
    [allowFrontCamera, devices],
  );

  const canUseFlash = device?.hasFlash;
  const canUseLowLightBoost = device?.supportsLowLightBoost;
  const canUseHDR = format?.supportsPhotoHdr;

  const onChangeCameraPositionButtonPress = useCallback(() => {
    setCameraPosition((prev) => (prev === 'front' ? 'back' : 'front'));
  }, []);

  if (device === null || device === undefined) {
    return <FullScreenLoader loaderColor={Color.PrimaryInitial} overlayColor={Color.BlackInitial} />;
  }

  return (
    <>
      <ReanimatedCamera
        ref={cameraRef}
        style={styles.cameraView}
        device={device}
        isActive={isActive}
        photo
        animatedProps={animatedProps}
        enableZoomGesture={enableZoomGesture}
        orientation="portrait"
        photoHdr={isHdrActive}
        lowLightBoost={isLowLightActive}
        enableLocation
        {...(isAndroid && { format: format })}
      />

      <View style={[styles.cameraControlButtonContainer, { top: topInset + 10 }]}>
        {canChangeCameraPosition && (
          <IconButton
            icon={<FontAwesomeIcon icon={faCameraRotate} size={22} />}
            type={'flat'}
            onPress={onChangeCameraPositionButtonPress}
            iconContainerStyle={styles.cameraControlButton}
            iconStyle={styles.cameraControlButtonIcon}
          />
        )}
        {canUseFlash && (
          <IconButton
            icon={getFlashModeIcon(flash)}
            type={'flat'}
            iconContainerStyle={styles.cameraControlButton}
            iconStyle={styles.cameraControlButtonIcon}
            onPress={onFlashButtonPress}
          />
        )}
        {canUseHDR && (
          <IconButton
            icon={<HDRLogo width={22} height={22} />}
            type={'flat'}
            iconContainerStyle={styles.cameraControlButton}
            iconStyle={styles.cameraControlButtonIcon}
            onPress={handleHdrSwitch}
          />
        )}

        {canUseLowLightBoost && (
          <IconButton
            icon={<FontAwesomeIcon icon={faMoon} size={22} />}
            type={'flat'}
            iconContainerStyle={styles.cameraControlButton}
            iconStyle={styles.cameraControlButtonIcon}
            onPress={handleLowLightBoost}
          />
        )}
      </View>

      <Pressable
        style={[styles.cameraControlTakePhotoButton, styles.cameraButtonsContainer, { bottom: bottomInset + 40 }]}
        onPress={onCameraPhotoButtonPress}>
        <Reanimated.View style={[styles.takePhotoButton, buttonFillAnimationStyle]} />
      </Pressable>
    </>
  );
};

const styles = StyleSheet.create({
  cameraView: styleUtils.view({
    ...StyleSheet.absoluteFillObject,
  }),
  cameraButtonsContainer: styleUtils.view({
    position: 'absolute',
    alignSelf: 'center',
    alignItems: 'center',
    justifyContent: 'center',
    flexDirection: 'row',
  }),
  cameraControlButtonContainer: styleUtils.view({
    position: 'absolute',
    right: 12,
    alignItems: 'flex-end',
    justifyContent: 'flex-end',
  }),
  cameraControlButton: styleUtils.view({
    width: 40,
    height: 40,
    borderCurve: 'circular',
    borderRadius: 20,
    alignItems: 'center',
    justifyContent: 'center',
    backgroundColor: Color.White35,
    marginBottom: 8,
  }),
  cameraControlTakePhotoButton: styleUtils.view({
    borderColor: Color.WhiteInitial,
    borderWidth: 4,
    borderCurve: 'circular',
    borderRadius: 40,
    width: 80,
    height: 80,
    marginHorizontal: 20,
    alignItems: 'center',
    justifyContent: 'center',
  }),
  takePhotoButton: styleUtils.view({
    backgroundColor: Color.White50,
    borderCurve: 'circular',
  }),
  cameraControlButtonIcon: styleUtils.image({
    resizeMode: 'contain',
    tintColor: Color.BlackInitial,
  }),
  cameraControlLeftButtonIcon: styleUtils.image({
    resizeMode: 'contain',
    tintColor: Color.WhiteInitial,
  }),
});

Relevant log output

{"tags": {"ApertureValue": 1.69, "BrightnessValue": 0, "ColorSpace": 1, "ComponentsConfiguration": [644, 4], "Compression": 6, "DateTime": "2024:04:03 14:48:03", "DateTimeDigitized": "2024:04:03 14:48:03", "DateTimeOriginal": "2024:04:03 14:48:03", "DigitalZoomRatio": 1, "ExifVersion": [368, 4], "ExposureBiasValue": 0, "ExposureMode": 0, "ExposureProgram": 0, "ExposureTime": 0.030004, "FNumber": 1.8, "Flash": 24, "FlashpixVersion": [476, 4], "FocalLength": 4.05, "FocalLengthIn35mmFilm": 28, "GPSAltitude": 0.0999, "GPSAltitudeRef": 0, "GPSDateStamp": "2024:04:03", "GPSImgDirectionRef": "133", "GPSLatitude": 48.948969999999996, "GPSLatitudeRef": "N", "GPSLongitude": 2.259605, "GPSLongitudeRef": "E", "GPSProcessingMethod": "gps", "GPSSpeed": 0, "GPSSpeedRef": "K", "GPSTimeStamp": "04:42:24", "ISOSpeedRatings": [632, 2], "ImageLength": 3840, "ImageWidth": 2160, "InteroperabilityIndex": "R98", "JPEGInterchangeFormat": 1883, "JPEGInterchangeFormatLength": 40576, "LightSource": 255, "Make": "OPPO", "MakerNote": "{\"nightFlag\":\"0\",\"nightMode\":\"0\",\"iso\":\"0\",\"expTime\":\"0\",\"featuretype\":\"0\",\"hdrscope\":\"0\",\"aiDeblur\":\"0\"}", "MaxApertureValue": 0, "MeteringMode": 2, "Model": "CPH2477", "Orientation": 0, "PixelXDimension": 2160, "PixelYDimension": 3840, "ResolutionUnit": 2, "SceneCaptureType": 0, "SensingMethod": 0, "ShutterSpeedValue": 5.044, "Software": "MediaTek Camera Application", "SubSecTime": "229", "SubSecTimeDigitized": "104", "SubSecTimeOriginal": "104", "WhiteBalance": 0, "XResolution": 72, "YCbCrPositioning": 2, "YResolution": 72}, "uri": "file:///data/user/0/com.geowatchapp/cache/mrousavy-5561020675257692647.jpg"}

Camera Device

{
  "formats": [],
  "sensorOrientation": "landscape-left",
  "hardwareLevel": "full",
  "maxZoom": 5,
  "minZoom": 1,
  "maxExposure": 4,
  "supportsLowLightBoost": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": false,
  "minFocusDistance": 8,
  "minExposure": -4,
  "name": "0 (BACK) androidx.camera.camera2",
  "hasFlash": true,
  "hasTorch": true,
  "position": "back",
  "id": "0"
}
 LOG  {
  "formats": [],
  "sensorOrientation": "landscape-left",
  "hardwareLevel": "full",
  "maxZoom": 5,
  "minZoom": 1,
  "maxExposure": 4,
  "supportsLowLightBoost": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": false,
  "minFocusDistance": 8,
  "minExposure": -4,
  "name": "0 (BACK) androidx.camera.camera2",
  "hasFlash": true,
  "hasTorch": true,
  "position": "back",
  "id": "0"
}
 LOG  {
  "formats": [],
  "sensorOrientation": "landscape-left",
  "hardwareLevel": "full",
  "maxZoom": 5,
  "minZoom": 1,
  "maxExposure": 4,
  "supportsLowLightBoost": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": false,
  "minFocusDistance": 8,
  "minExposure": -4,
  "name": "0 (BACK) androidx.camera.camera2",
  "hasFlash": true,
  "hasTorch": true,
  "position": "back",
  "id": "0"
}

Device

OPPO A17

VisionCamera Version

4.0.0-beta.11

Can you reproduce this issue in the VisionCamera Example app?

I didn't try (⚠️ your issue might get ignored & closed if you don't try this)

Additional information

• [ ] I am using Expo
• [ ] I have enabled Frame Processors (react-native-worklets-core)
• [X] I have read the Troubleshooting Guide
• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar issues in this repository and found none.

[mrousavy]

Well it does start updating location, but the initial value is set to the cached value. It should give an update within the first 5 seconds of running, does that never happen? Or is 5 seconds just too slow for your usecase?

[idrisssakhi]

yes, it does update the location after a while, didn't measure the time. Any way, in my use case I need to get the real actual location. I used exify for android to write metadata directly. if it's a lot of time to improve we can close this, i think there are more important issues than this

[mrousavy]

Can you maybe help debug this a bit? I'm not sure what values I need to put in here: https://github.com/mrousavy/react-native-vision-camera/blob/5a350c6dc9f8328cadf8982e59b4ae69146952ad/package/android/src/main/java/com/mrousavy/camera/core/MetadataProvider.kt#L38-L40

Currently those values are:

• UPDATE_INTERVAL_MS = 5000L (= 5 seconds)
• UPDATE_DISTANCE_M = 5f (= 5 meters)

..aren't that reasonable values? If I set both to 0 it will stream location quite aggressively, no?

There is also a method to imperatively get the current location (LocationManager.getCurrentLocation) but this is API level 30+.

[idrisssakhi]

@mrousavy seems good I made a sort of loader when opening camera on android to extract location & if I wait a bit before taking the first image the location is good. we can close this