timeout while extracting gopro metadata from mp4 file

[time4tea]

Created from comment in: https://github.com/time4tea/gopro-dashboard-overlay/issues/35

It's timeout here: https://github.com/time4tea/gopro-dashboard-overlay/blob/main/gopro_overlay/ffmpeg.py#L182

This is the result of running with unmodified code (with the timeout set):

$ gopro-dashboard.py 2022-08-21_ride.mp4 2022-08-21_ride_overlay.mp4
Starting gopro-dashboard version 0.58.0
Input file has size Dimension(x=3840, y=2160)
Timer(loading timeseries - Called: 1, Total: 10.24270, Avg: 10.24270, Rate: 0.10)
Timer(program - Called: 1, Total: 10.24276, Avg: 10.24276, Rate: 0.10)
Traceback (most recent call last):
  File "/home/zebradil/archive/tmp/tools/venv/bin/gopro-dashboard.py", line 99, in <module>
    gopro_frame_meta = framemeta_from(
  File "/home/zebradil/archive/tmp/tools/venv/lib/python3.10/site-packages/gopro_overlay/framemeta.py", line 242, in framemeta_from
    gpmd_from = load_gpmd_from(filepath)
  File "/home/zebradil/archive/tmp/tools/venv/lib/python3.10/site-packages/gopro_overlay/ffmpeg.py", line 182, in load_gpmd_from
    result = run(cmd, capture_output=True, timeout=10)
  File "/home/zebradil/archive/tmp/tools/venv/lib/python3.10/site-packages/gopro_overlay/ffmpeg.py", line 20, in run
    return subprocess.run(cmd, check=True, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 503, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "/usr/lib/python3.10/subprocess.py", line 1152, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.10/subprocess.py", line 2004, in _communicate
    self._check_timeout(endtime, orig_timeout, stdout, stderr)
  File "/usr/lib/python3.10/subprocess.py", line 1196, in _check_timeout
    raise TimeoutExpired(
subprocess.TimeoutExpired: Command '['ffmpeg', '-hide_banner', '-y', '-i', '2022-08-21_ride.mp4', '-codec', 'copy', '-map', '0:1', '-f', 'rawvideo', '-']' timed out after 10 seconds

Running ffmpeg -hide_banner -y -i 2022-08-21_ride.mp4 -codec copy -map 0:1 -f rawvideo - directly produces a lot of output into the stdout.

[time4tea]

@zebradil - I created this based on your comment. I've never seen this error. Can you tell me a bit about your setup? Do you run this command accessing the sdcard directly, or do you copy the mp4 file to your local disk first? What type of computer do you run this on? Any other information you think might be relevant? Thanks!

[time4tea]

Is this still a problem for you?

[YouthfulRicker]

It just happened to me as well, exactly the same error message: subprocess.TimeoutExpired: Command '['ffmpeg', '-hide_banner', '-y', '-i', PosixPath('GH010076.MP4'), '-codec', 'copy', '-map', '0:2', '-f', 'rawvideo', '-']' timed out after 10 seconds I have my video on a external HDD and am trying to export the video to the same location with a -dashboard at the end of the file (before the filetype tag). Also sidenote, I installed it via pip on a Mac running MacOS Big Sur.

[time4tea]

Please could you try copying the file to an internal drive and see if it fixes the problem.

I have a suspicion that these problems are caused by data transfer rate.

[YouthfulRicker]

Ok, after copying the file to my mac's internal ssd, lo and behold it works how I imagine it should!:

Starting gopro-dashboard version 0.73.0
    Timer(GPMD - Called: 1, Total: 1.87310, Avg: 1.87310, Rate: 0.53)
    Timer(extract GPS - Called: 1, Total: 1.06084, Avg: 1.06084, Rate: 0.94)
Request for data at time Timeunit ms=1571004.043, after end of metadata, returning last item
    Timer(extract ACCL - Called: 1, Total: 2.22369, Avg: 2.22369, Rate: 0.45)
Request for data at time Timeunit ms=-49.226, before start of metadata, returning first item
Request for data at time Timeunit ms=1571004.043, after end of metadata, returning last item
    Timer(extract GRAV - Called: 1, Total: 0.79876, Avg: 0.79876, Rate: 1.25)
Request for data at time Timeunit ms=-49.226, before start of metadata, returning first item
Request for data at time Timeunit ms=1571004.043, after end of metadata, returning last item
    Timer(extract CORI - Called: 1, Total: 1.12882, Avg: 1.12882, Rate: 0.89)
Timer(parsing - Called: 1, Total: 7.09024, Avg: 7.09024, Rate: 0.14)
Timer(loading timeseries - Called: 1, Total: 9.05051, Avg: 9.05051, Rate: 0.11)
Generating overlay at Dimension(x=1920, y=1080)
Timeseries has 3144 data points
Processing....
Timer(processing - Called: 1, Total: 1.34920, Avg: 1.34920, Rate: 0.74)
FFMPEG Output is in /var/folders/lk/jqrdwhnx4cv7pr5y40p8_5d40000gn/T/tmpbeliqrm3
Timelapse Factor = 14.980
Layout -> Include component 'date_and_time' = True
Layout -> Include component 'gps_info' = True
Layout -> Include component 'big_mph' = True
Layout -> Include component 'gradient_chart' = True
Layout -> Include component 'gradient' = True
Layout -> Include component 'altitude' = True
Layout -> Include component 'temperature' = True
Layout -> Include component 'cadence' = True
Layout -> Include component 'heartbeat' = True
Layout -> Include component 'moving_map' = True
Layout -> Include component 'journey_map' = True
Executing 'ffmpeg -y -hide_banner -loglevel info -i Documents/Driving Time Lapses/GH010076.MP4 -f rawvideo -framerate 10.0 -s 1920x1080 -pix_fmt rgba -i - -filter_complex [0:v][1:v]overlay -vcodec libx264 -preset veryfast Documents/Driving Time Lapses/GH010076-dash.MP4'
Render: 1048 [ 99%]  [  5.4/s] |############################################################################################################################################################### | ETA:   0:00:00Finished drawing frames. waiting for ffmpeg to catch up
Render: 1049 [100%]  [  5.4/s] |################################################################################################################################################################| Time:  0:03:15
Timer(image to bytes - Called: 1,049, Total: 17.90877, Avg: 0.01707, Rate: 58.57)
Timer(writing to ffmpeg - Called: 1,049, Total: 6.51752, Avg: 0.00621, Rate: 160.95)
Timer(drawing frames - Called: 1,049, Total: 166.70469, Avg: 0.15892, Rate: 6.29)
Timer(program - Called: 1, Total: 205.66148, Avg: 205.66148, Rate: 0.00)

I would say theoretically that it could be a problem with my HDD's enclosure being of USB 2 connection, and it could also just be the use of an external drive. Either way the superior speed SSD worked. I'll quickly try with a USB 3 External HDD

[YouthfulRicker]

Update: My External USB 3 HDD did not work and caused the initial problem.

[time4tea]

Thanks for the information! - That's really useful. I don't want to penalise people for using slower devices... but at the same time the rendering speed will be really slow if reading and writing to an external drive. External drives have many potential issues: slow transfer (max theoretical speed of USB2.0 is 480Mb/s - so a 4Gb file will take 1 min to read, and devices will not achieve this in practice), lower spec drive - external drives tend to have lower rotation speed and slower seek time. I think i could usefully improve the error messages though...

[time4tea]

Also... thank you for your nice comments! :-)

[YouthfulRicker]

Thanks for the information! - That's really useful. I don't want to penalise people for using slower devices... but at the same time the rendering speed will be really slow if reading and writing to an external drive.

I understand, I personally am fine with doing a temporary transfer of data to my local drive.

And I agree with the error message addition, it would make sense to maybe have on the next line a simple message like "This external drive doesn't have the required speed capabilities" or something.

I believe you can now add the possibly fixed tag to this post now.

[YouthfulRicker]

Also... thank you for your nice comments! :-)

No problem, this program has genuinely saved my time and money. And it is really well done too!

[time4tea]

@YouthfulRicker - if you were able to share a still or a short video that you created with the program, that would be really fab. At the same don't please don't feel obliged! - If its a video, please use WeTransfer or similar and send link to "gopro-overlay (at) time4tea.net"

[YouthfulRicker]

@YouthfulRicker - if you were able to share a still or a short video that you created with the program, that would be really fab. At the same don't please don't feel obliged! - If its a video, please use WeTransfer or similar and send link to "gopro-overlay (at) time4tea.net"

For sure! I'll send it when possible.

[time4tea]

Added explanatory message on timeout. Fixed in 0.76.0