--capture-sound does not capture sound (from Canon EOS R)

[criticaltv]

Describe the bug

When passing in the --capture-sound flag to gphoto2, to capture from Canon EOS R, no sound is captured.

Name the camera

studio@cibulka:~$ gphoto2 --auto-detect
Model                          Port                                            
----------------------------------------------------------
Canon EOS R                    usb:004,027     

libgphoto2 and gphoto2 version

gphoto2 2.5.23

Copyright (c) 2000-2019 Marcus Meissner and others

gphoto2 comes with NO WARRANTY, to the extent permitted by law. You may
redistribute copies of gphoto2 under the terms of the GNU General Public
License. For more information about these matters, see the files named COPYING.

This version of gphoto2 is using the following software versions and options:
gphoto2         2.5.23         gcc, popt(m), exif, cdk, aa, jpeg, readline
libgphoto2      2.5.25         standard camlibs (SKIPPING lumix), gcc, ltdl, EXIF
libgphoto2_port 0.12.0         iolibs: disk ptpip serial usb1 usbdiskdirect usbscsi, gcc, ltdl, EXIF, USB, serial without locking

To Reproduce

1. Run the following command:

gphoto2 --capture-movie --capture-sound --debug --debug-logfile=debug.log

2. Press ctrl-c to finish the capture
3. Play the movie.mjpg - inside zip.
4. Unexpected Behaviour: observe that no sound is stored in the video.
5. Observe logs per this https://github.com/videoDAC/raspberrypi/blob/master/debug.log.zip?raw=true

[msmeissn]

We currently have not implemented capture-sound anywhere.

For EOS R the "capture-movie" is just a preview stream.

Capturing a real movie currently will happen only on the card, and has the file transfered later.

[criticaltv]

Thank you for your response, but I don't know if this addresses my issue.

Perhaps it would help to explain my use case:

I am currently using gphoto2 --stdout --capture-movie and pipe it into ffmpeg for livestreaming via rtmp, like this:

gphoto2 --stdout --capture-movie --capture-sound | ffmpeg -i - -vcodec h264 -f lavfi -i anullsrc -c:v libx264 -b:v 1000k -c:a aac -x264-params keyint=60 -f flv rtmp://yourRTMPserver:1935

Currently, I need to generate the audio for the livestream using anullsrc in ffmpeg - as my RTMP endpoint requires video and sound to be presented.

But the camera does present audio via USB, which is available in Canon EOS Utility's Liveview mode. This is synced with the video, so is super-useful for livestreaming.

The dream scenario would be for gphoto2 to also capture audio from the EOS R (and any other camera which presents audio), and layer it into the video (in sync) - which can then be instantly livestreamed.

Then if / when Canon update the firmware to send 4K (over USB-C) to gphoto2, then we have an incredible potential setup... I'm livestreamer in this thread: https://community.usa.canon.com/t5/Software/Feature-Request-EOS-Webcam-Utility-to-present-Sound-Interface/m-p/314295#M10463

Also, on a side note - does gphoto2 do anything with capturing content over WiFi? This would be the ultimate in wireless livestreaming camera...

[chrishobcroft]

The Camera Connect app on Android is able to detect live sound levels from the EOS R microphone over WiFi:

Also, live sound levels are displayed in Canon EOS Utility's "Liveview" feature on a Mac, when connecting over WiFi or USB (no screenshot available rn).

So if these apps are able to receive audio data from the camera over Wifi, could this be used by the --capture-sound function in libgphoto2 / gphoto2?

cc. https://github.com/gphoto/gphoto2/issues/357

[msmeissn]

is there any indication it transfers audio, or just reports the levels? Might be something I am not aware of yet.

[chrishobcroft]

I was considering this also.

There doesn't appear to be a way to monitor the audio levels using headphones, via the app or Eos utility.

But the sound level monitor does appear to be displaying quite granular levels of information, and be quite sensitive and responsive.

I guess it is possible that the camera is only sending two streams of live data (Left side, Right side) to allow the app to display -dB levels... but it just doesn't feel that way... why would someone writing firmware for the camera bother to write a converter from the raw sampled audio into a data feed specific to EOS Utility? Or maybe they did, and that's all it does. I mean, it is Canon after all.

How could we check? How can we query the interface?

If we could get synchronised audio and video from the camera, even over USB, it would be an important step towards delivering https://github.com/gphoto/gphoto2/issues/354

What do you think?

[chrishobcroft]

Here is how EOS Utility presents the audio information retrieved from the camera:

https://vimeo.com/449660611

Screenshot from video:

[chrishobcroft]

Hi @msmeissn how are you?

Interested in progressing this in relation to https://github.com/gphoto/gphoto2/issues/357.

Do you have any advice?