smserver Apple's proprietary file formats behave strangely sometimes (MOV, possibly HEIC, m4v, caf, m4a, etc.)

Apple's proprietary file formats behave strangely sometimes (MOV, possibly HEIC, m4v, caf, m4a, etc.)

Open sabogalc opened this issue 3 years ago • 11 comments

109396140-23a56400-78fe-11eb-8a13-ae24a61cf086 ^ I received this error when trying to play some MOV videos in FireFox 109396151-34ee7080-78fe-11eb-961c-0ee1f63bf9bc and then one particular MOV file only played as an audio file.

Here is the link to Aziz's first comment on the issue, and all the comments below that one are helpful/insightful too.

==== Ignore this rant section, it isn't relevant to the issue, I'm just upset. It's at the bottom to not interfere with the ticket ==== In a perfect world, everyone is using industry standards like mp3 and mp4 so that things Just Work™, but at Apple, they like to “ Think Different”. This MOV problem shouldn't exist, and the quotes I used in the last sentence should look upright like "this", just like they do on any device that isn't made by Apple. We shouldn't need to use workaround software to use iMessage on other devices (not to say Apple is the only company guilty of this). I understand that exclusives are good and they serve to sell more products. I wouldn't want Halo Infinite to be playable on a PS5, but if I have something as simple as a Blu-Ray movie disk that contains nothing more than a simple video, that disk should be playable on both an Xbox and a PlayStation (and thankfully it is!). If Apple wants to make their live photos and MeMojis only work as intended on their devices, then fine. And if they want to push for standards such as HEIC, that's fine too. But when they use something proprietary like MOV, there should be enough licensing information available to at the very least ensure that their format will behave like any other video file if it gets sent elsewhere (hmm but if only there were a format that already does that). I have similar things to say about lightning vs USB-C and the proprietary SSD's on the older MacBooks (which are now soldered on so it doesn't make a difference). Anyways, sorry for that. I'm done now 😃 ==== Rant over ====

Mar 02 '21 17:03 sabogalc

data?path=Attachments/46/06/3DFC540E-5B42-42D1-A064-02D8740F21F5/63658281952__44A78AC2-8827-4B97-8513-BF81F0227D78.fullsizerender.heic Here is an HEIC image that didn't display for me, like in issue #82

Mar 04 '21 22:03 sabogalc

So SMServer should automatically convert heic images to png, then send that png data to the browser so that you can still view them. It doesn't yet do this kinda thing for caf or mov files, but I'll look into that.

However, I've noticed a weird issue where some images will show up fine while I'm debugging the app (specifically, when I'm using the version of the app that is transferred over to my phone when I do debugging on Xcode), but then they won't show up at all as soon as I package the app for distribution. It isn't a release vs debug configuration issue, since this issue still persists when I use a release configuration while debugging. It isn't a permissions issue, since it shows up regardless of how the app is installed (e.g. as an app store app or as a system app) and regardless of who's running it (mobile vs root). So I can't quite figure out what's causing it, but this may be causing/related to the heic image not rendering.

Mar 05 '21 18:03 itsjunetime

I just tested the HEIC image sent from both SMServer and WebMessage, and I was unable to view it in either HEIC WebMessage HEIC That’s a really weird bug with the release thing, I wonder what could be causing that

Mar 05 '21 18:03 sabogalc

The code for handling attachment requests in WebMessage is basically identical to the same code in SMServer, so they probably suffer from the same issues and it makes sense that the same heic files that don't work in one would also not work in the other.

Mar 05 '21 18:03 itsjunetime

Would you be able to provide an example of a specific mov file that doesn't play in your browser? All the ones that I've tried run just fine in both firefox and chrome (permitting a fast enough internet connection, of course).

Mar 05 '21 23:03 itsjunetime

Any video taken through the Camera roll app will have this issue. I was able to get this fixed with the AVAssetExportSession class in my test build. However, it does take up some time and should be opt-in as it can add a good amount of processing.

The heic image issue is strange, and I am testing this with the same file used by @CSab6482 with no luck of reproducing.

Mar 06 '21 09:03 sgtaziz

Any video taken through the Camera roll app will have this issue. I was able to get this fixed with the AVAssetExportSession class in my test build. However, it does take up some time and should be opt-in as it can add a good amount of processing.

The heic image issue is strange, and I am testing this with the same file used by @CSab6482 with no luck of reproducing.

I'll jailbreak my iPad and use it as an SMServer/WebMessage host device and try testing the HEIC image on there too to see if it's a problem with my iPhone 6S.

Mar 06 '21 15:03 sabogalc

I forgot to update this. It's a strange situation. Sending an HEIC from SMServer shows up properly on all other devices except for the host device. It converts it to a jpeg for everyone else. iPhone I sent the first HEIC image from my instance of SMServer that is hosted on my iPhone, and the second one from my iPad SMServer. As you can see, my iPhone server displays the iPad version of the image, but not the one that it sent. iPad This behavior is also mirrored on my iPad. Let me know if more information is needed.