Images sampled from GoPro split videos repeat the timestamps for each video segment

[RudyTheDev]

Basic information

• Release version: main (but with a fix applied first from https://github.com/mapillary/mapillary_tools/issues/503)
• System: Windows 10 Pro x64
• Capture Device: GoPro HERO8 Black HD8.01.02.50.00

Steps to reproduce behavior

1. Have a video recorded on GoPro Hero 8 long enough to be split into 2+ files, i.e. videos/GH01####.MP4, videos/GH02####.MP4, etc.
2. Run image sampling from video, for example mapillary_tools sample_video "videos/" "images/" --video_sample_interval 1

Expected behavior

images/GH01####.MP4/ folder has image files with EXIF timestamps 2022-05-28 18:00:00, 2022-05-28 18:00:01, etc. images/GH02####.MP4/ folder has image files with EXIF timestamps 2022-05-28 18:09:10, 2022-05-28 18:09:11, etc. etc.

Actual behavior

images/GH01####.MP4/ folder has image files with EXIF timestamps 2022-05-28 18:00:00, 2022-05-28 18:00:01, etc. images/GH02####.MP4/ folder has image files with EXIF timestamps 2022-05-28 18:00:00, 2022-05-28 18:00:01, etc. etc.

Additional information

I believe the problem is that my GoPro 8's split video segments all have the same stream/track creation date metadata. For instance, both videos in the above example would have "Media Create Date" set to "2022:05:28 18:00:00". So my GoPro doesn't actually record each segment's creation time, but only the creation time of the whole recording.

I tried various parameters and combinations, but didn't get anything to work. I tried seeing if there was some place where the code would handle something like this, but also didn't see anything.

What I did locally to fix this (https://github.com/RudyTheDev/mapillary_tools/commit/75620445aca75f662dab6f3f94fe1e0a498feff3) is to keep the total elapsed time and offset every consecutive video by that amount. This is a quick-and-dirty fix that doesn't know anything about the video sequence or GoPro logic, but this does produce the correct/expected timestamps for me.

[ptpt]

Thanks for the info. Is there any way to identify if segments belong to one video other than the name pattern? Could you check the videos with ffprobe -show_streams -print_format json -show_format videos/GH01####.MP4 and see if we can find something there.

[RudyTheDev]

Oh, ffprobe actually shows things that exiftool doesn't. I didn't realize that. I'll dump the full results from a sequence I have below, but here's the only useful thing I see from first, second (15 skipped) and last video files:

'GH010137.MP4':
  Metadata:
    creation_time   : 2022-05-31T17:49:29.000000Z
  Duration: 00:08:51.54
  Stream #0:0
    Metadata:
      creation_time   : 2022-05-31T17:49:29.000000Z
      timecode        : 17:48:25:16

'GH020137.MP4':
  Metadata:
    creation_time   : 2022-05-31T17:49:29.000000Z
  Duration: 00:08:51.54
  Stream #0:0
    Metadata:
      creation_time   : 2022-05-31T17:49:29.000000Z
      timecode        : 17:57:16:16

'GH180137.MP4':
  Metadata:
    creation_time   : 2022-05-31T17:49:29.000000Z
  Duration: 00:07:45.08
  Stream #0:0
    Metadata:
      creation_time   : 2022-05-31T17:49:29.000000Z
      timecode        : 20:18:52:16

Looks like creation_time for the container and all streams is the same, but the timecode for embedded streams is per-file.

I don't see anything to uniquely identify a sequence of videos besides the creation date. Without a file name, you can't even directly tell which one in the sequence it is. I also compared the output between two different sequences and there is no difference besides the creation time and time code.

Here is the full probe dump: https://github.com/RudyTheDev/mapillary_tools/commit/7b290c15b64b39857de4244a0cb6400e25bbf38c

[ptpt]

Hey @RudyTheDev could you try out the latest commit and see if it works. Feel free to reopen if it does not.

[RudyTheDev]

The current main revision gives me this when run:

Traceback (most recent call last):
  File "C:\Python\Python310\Scripts\mapillary_tools-script.py", line 33, in <module>
    sys.exit(load_entry_point('mapillary-tools==0.9.0', 'console_scripts', 'mapillary_tools')())
  File "C:\Python\Python310\Scripts\mapillary_tools-script.py", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "C:\Python\Python310\lib\importlib\metadata\__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "C:\Python\Python310\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\Python\Python310\lib\site-packages\mapillary_tools-0.9.0-py3.10.egg\mapillary_tools\commands\__init__.py", line 1, in <module>
  File "C:\Python\Python310\lib\site-packages\mapillary_tools-0.9.0-py3.10.egg\mapillary_tools\commands\process.py", line 5, in <module>
  File "C:\Python\Python310\lib\site-packages\mapillary_tools-0.9.0-py3.10.egg\mapillary_tools\process_geotag_properties.py", line 15, in <module>
  File "C:\Python\Python310\lib\site-packages\mapillary_tools-0.9.0-py3.10.egg\mapillary_tools\geotag\geotag_from_blackvue.py", line 13, in <module>
  File "C:\Python\Python310\lib\site-packages\mapillary_tools-0.9.0-py3.10.egg\mapillary_tools\geotag\blackvue_utils.py", line 9, in <module>
  File "C:\Python\Python310\lib\site-packages\mapillary_tools-0.9.0-py3.10.egg\mapillary_tools\geotag\simple_mp4_parser.py", line 234, in <module>
TypeError: Prefixed.__init__() got an unexpected keyword argument 'includelength'

[ptpt]

What "construct" version are you using? Check:

python3 -m pip show construct

If it is lower than the specified version construct>=2.10.0,<3.0.0 in requirements.txt, upgrade it with python3 -m pip install -U construct

[RudyTheDev]

I checked the version and it was 2.8.8. I ran the upgrade and it upgraded to 2.10.68. It complained that pip was incompatible, so I upgraded it from 22.0.4 to 22.2.2. It runs now. Sorry if this is some basic stuff, I really don't know anything about Python and the setup.py didn't say anything was wrong (that I saw anyway).

I don't have a full sequence right now to sample; I tried to sample 2 consecutive videos, but unfortunately it looks like the image batches still restart the timestamps for each video file. I'll try a proper sequence tomorrow.

[RudyTheDev]

I ran on it on 4-file long sequence.

The differences between video files in ffprobe are as before - timecode in stream metadata is different. (The only other differences are tiny duration, bitrate, size and frame count.)

The creation timestamp of all streams in each file is the same:

creation_time   : 2022-08-22T17:42:05.000000Z

Time codes of all the individual streams in each file are:

timecode        : 17:41:01:56
timecode        : 17:46:20:56
timecode        : 17:51:39:56
timecode        : 17:56:58:56

The durations of video files are:

Duration: 00:05:19.34
Duration: 00:05:19.32
Duration: 00:05:19.32
Duration: 00:02:34.29

There at 1064, 1064, 1064, and 514 files sampled from the videos at 0.3 (mapillary_tools --verbose sample_video "videos/" "images/" --video_sample_interval 0.3).

EXIF timestamps of first images from file sequence are:

Date/Time: 2022-08-22 17:36:45.681
Date/Time: 2022-08-22 17:36:45.681
Date/Time: 2022-08-22 17:36:45.681
Date/Time: 2022-08-22 17:39:30.712

EXIF timestamps of last images from file sequence are:

Date/Time: 2022-08-22 17:42:04.581
Date/Time: 2022-08-22 17:42:04.581
Date/Time: 2022-08-22 17:42:04.581
Date/Time: 2022-08-22 17:42:04.612

[ptpt]

Thanks for the update.

I wasn't clear on what the PR fixes. https://github.com/mapillary/mapillary_tools/pull/529 doesn't fix video segment creation times. Instead it forces the samples and the GPS trace to use the same clock, so that the samples can be interpolated on the right locations along the GPS trace.

I will address the creation timestamp issue in another PR.