Live videos don't play

[meichthys]

The live photos feature has made Memories so much more usable (No more duplicate photo/videos!). 👏

It doesn't bother me much, but i did notice that the live video doesn't seem to play when clicking on (or hovering) over the live video icon. I do see the following in the console logs:

Media resource https://nextcloud.mydomain.com/apps/memories/api/video/livephoto/611885?etag=38076727bb7da1dad8b958ec937f19ee&liveid=CAF8D2E2-2B80-46D1-8E6B-31742E3E241F could not be decoded. [memories](https://nextcloud.mydomain.com/apps/memories/)
Media resource https://nextcloud.mydomain.com/apps/memories/api/video/livephoto/611885?etag=38076727bb7da1dad8b958ec937f19ee&liveid=CAF8D2E2-2B80-46D1-8E6B-31742E3E241F could not be decoded, error: Error Code: NS_ERROR_DOM_MEDIA_DEMUXER_ERR (0x806e000c)
Details: virtual RefPtr<MP4Demuxer::InitPromise> mozilla::MP4Demuxer::Init(): No MP4 audio () or video () tracks

[pulsejet]

Sounds like mpv4, which nobody supports. Can you provide a sample? We might need to transcode these :( I assume this is Apple? https://bugzilla.mozilla.org/show_bug.cgi?id=1427278

[meichthys]

These are all Apple .mov files. Here's a sample:

https://user-images.githubusercontent.com/10717998/203460759-c2e4dbc3-269d-40b5-9609-14054822cb7e.mov

[pulsejet]

I need the JPEG part too

[meichthys]

[pulsejet]

Looks like this is HEVC. Now that sucks.

[meichthys]

Ah ok. even though the extension is .jpg?

[pulsejet]

Not the JPEG. The MOV is HEVC. No browser supports that, so the only way to do this is transcode. It should be possible (and hopefully easy) to utilize the same transcoding backend when enabled ...

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '203460759-c2e4dbc3-269d-40b5-9609-14054822cb7e.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2022-11-21T21:49:32.000000Z
    com.apple.quicktime.location.accuracy.horizontal: 75.864683
    com.apple.quicktime.live-photo.auto: 1
    com.apple.quicktime.content.identifier: 021842E6-D17A-4C62-BC13-B2521961DF0B
    com.apple.quicktime.live-photo.vitality-score: 0.939850
    com.apple.quicktime.live-photo.vitality-scoring-version: 0
    com.apple.quicktime.location.ISO6709: +42.4218-075.5919+453.100/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone 12 mini
    com.apple.quicktime.software: 16.1.1
    com.apple.quicktime.creationdate: 2022-11-21T16:49:32-0500
  Duration: 00:00:02.92, start: 0.000000, bitrate: 12236 kb/s
    Stream #0:0(und): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, smpte170m/smpte432/bt709), 1920x1440, 11446 kb/s, 29.13 fps, 59.94 tbr, 600 tbn, 600 tbc (default)
    Metadata:
      creation_time   : 2022-11-21T21:49:32.000000Z
      handler_name    : Core Media Video
      encoder         : HEVC
    Stream #0:1(und): Audio: pcm_s16le (lpcm / 0x6D63706C), 44100 Hz, 1 channels, s16, 705 kb/s (default)
    Metadata:
      creation_time   : 2022-11-21T21:49:32.000000Z
      handler_name    : Core Media Audio
    Stream #0:2(und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
    Metadata:
      creation_time   : 2022-11-21T21:49:32.000000Z
      handler_name    : Core Media Metadata
    Stream #0:3(und): Data: none (mebx / 0x7862656D), 33 kb/s (default)
    Metadata:
      creation_time   : 2022-11-21T21:49:32.000000Z
      handler_name    : Core Media Metadata
    Stream #0:4(und): Data: none (mebx / 0x7862656D), 504 kb/s (default)
    Metadata:
      creation_time   : 2022-11-21T21:49:32.000000Z
      handler_name    : Core Media Metadata

[meichthys]

The Live Photos do seem to work for me now. However, on mobile, there does not seem to be a way to make Live Photos play. Clicking on the Live Photos icon does not seem to do anything on mobile.

[pulsejet]

The Live Photos do seem to work for me now.

Browser / platform? There's some flaky HEVC support here and there https://caniuse.com/hevc

However, on mobile, there does not seem to be a way to make Live Photos play. Clicking on the Live Photos icon does not seem to do anything on mobile.

Opening it should play the live photo. If it's HEVC again, won't work.

[meichthys]

I guess the hevc may still be not working - I'll retest after 4.9

The Live Photos that do play on web (desktop) don't seem to play when opening on iOS safari.

My cellular service is poor right now so it could be timing out trying to load the live video.

[pulsejet]

Yeah, we need to transcode these either way.

[invario]

Not the JPEG. The MOV is HEVC. No browser supports that, so the only way to do this is transcode. It should be possible (and hopefully easy) to utilize the same transcoding backend when enabled ...

Chrome (at least in Windows and Android) recently and quietly added HEVC.

My MP4 H265 videos recently started working in the browser.

[pulsejet]

Yup, it depends on both platform and hardware. https://caniuse.com/?search=hevc

[meichthys]

I'm just following up here to indicate that the Live Photos do play find for me now on Safari in 4.8.2, but the audio doesn't seem to be included in the live photo playback. Not really an issue for me, but it seems worth noting.