gpac
Feature Request: View mdat Data
I'm interested in viewing various mdat data, including:
- Image Items
- Metadata Items (grid, exif, & mime)
- Video Tracks
- Sample metadata from the saiz/saio boxes
Do these seem within scope for MP4Box?
The File Reader example demonstrates many features of MP4Box.js. Does that help you? If not, what's missing?
The File Reader doesn't show any information stored in the mdat box.
For example, it may show an item of type grid, but the grid data itself is hidden in the mdat:
aligned(8) class ImageGrid {
unsigned int(8) version = 0;
unsigned int(8) flags;
unsigned int FieldLength = ((flags & 1) + 1) * 16; // this is a temporary, non-parsable variable
unsigned int(8) rows_minus_one;
unsigned int(8) columns_minus_one;
unsigned int(FieldLength) output_width;
unsigned int(FieldLength) output_height;
}
Oh I see, yeah there is no parsing/decoding for mdat right now. You have access to the individual samples but no information as you described.
PRs always welcome as this is not something we have considered.
Okay, I'll try putting together a PR for this.
On a similar note, has there been consideration into turning MP4Box into a player? I see the main index.html file already plays video given a URL. Is there any reason why this couldn't be extended to play a local file?
The demos haven't been updated in a while but you can definitely use MP4Box.js as a player, either connect it to a video element with MSE (as seen here) or use onSamples callback and decode the samples via VideoDecoder.
If your concern is that why you can't just use MP4Box.js to play a file, what would MP4Box.js do differently than just using <video src="input.mp4" />
If your concern is why you can't just use MP4Box.js to play a file, what would MP4Box.js do differently than just using
<video src="input.mp4" />?
The HTML video tag and most other players have some limitations that MP4Box.js might be able to overcome. For example, say a file has a mix of multiple video tracks and images and the user wants to browse through and play each one.
There's also various codecs that aren't yet supported (J2K, HTJ2K, etc). Playing such files in MP4Box.js would require some decoding library which I'm not sure if there's interest in adding. I'm particularly interested in playing uncompressed MP4s/HEIFs which again isn't supported in most players.
You have onSamples callback that gives you samples per track. It shouldn't be the responsibility of this library to decode the samples. Maybe some sort of plugin based architecture could be employed to decode them if needed.
That requires some considerable planning though. Although, I can see the benefit of having that feature. We recently went through a complete rewrite. Once the dust settles on that stuff we could consider how such an architecture could be implemented.