TypeScript support?

[BTOdell]

trafficstars

Would be nice if this library had TypeScript support. I'll write up some basic definitions to get this started...

[BTOdell]

These TypeScript definitions can be used if you just need to library to perform moov box parsing.

declare module "mp4box" {

    interface MP4MediaTrack {
        id: number;
        created: Date;
        modified: Date;
        movie_duration: number;
        layer: number;
        alternate_group: number;
        volume: number;
        track_width: number;
        track_height: number;
        timescale: number;
        duration: number;
        bitrate: number;
        codec: string;
        language: string;
        nb_samples: number;
    }

    interface MP4VideoData {
        width: number;
        height: number;
    }

    interface MP4VideoTrack extends MP4MediaTrack {
        video: MP4VideoData;
    }

    interface MP4AudioData {
        sample_rate: number;
        channel_count: number;
        sample_size: number;
    }

    interface MP4AudioTrack extends MP4MediaTrack {
        audio: MP4AudioData;
    }

    type MP4Track = MP4VideoTrack | MP4AudioTrack;

    interface MP4Info {
        duration: number;
        timescale: number;
        fragment_duration: number;
        isFragmented: boolean;
        isProgressive: boolean;
        hasIOD: boolean;
        brands: string[];
        created: Date;
        modified: Date;
        tracks: MP4Track[];
    }

    export type MP4ArrayBuffer = ArrayBuffer & {fileStart: number};

    export interface MP4File {

        onMoovStart?: () => void;
        onReady?: (info: MP4Info) => void;
        onError?: (e: string) => void;

        appendBuffer(data: MP4ArrayBuffer): number;
        start(): void;
        stop(): void;
        flush(): void;

    }

    export function createFile(): MP4File;

    export { };

}

[cconcolato]

Yes, it probably would be a good idea to migrate to TypeScript, and thanks for the proposal, but not sure I'll have the time to work on it.

[oscartbeaumont]

It is probably worth adding types like what is above to DefinitelyTyped to save the amount of work it would take to do a complete rewrite while allowing Typescript users to work with this package.

[JohnWeisz]

Hi everyone, I'd also like to contribute to get typings done. My use case is audio decoding from mp4 containers, although that'll cover most of the video "reading" as well.

There is no need to rewrite the entire project from scratch if we just want to add typings. See my contribution to a similar project: https://github.com/eshaz/codec-parser/pull/23

All we need is the typings file(s), and pointing package.json to them.

[JohnWeisz]

Here are a few more in the meantime:

declare module "mp4box" {

    interface MP4MediaTrack
    {
        id: number;
        created: Date;
        modified: Date;
        movie_duration: number;
        movie_timescale: number;
        layer: number;
        alternate_group: number;
        volume: number;
        track_width: number;
        track_height: number;
        timescale: number;
        duration: number;
        bitrate: number;
        codec: string;
        language: string;
        nb_samples: number;
    }

    interface MP4VideoData
    {
        width: number;
        height: number;
    }

    interface MP4VideoTrack extends MP4MediaTrack
    {
        video: MP4VideoData;
    }

    interface MP4AudioData
    {
        sample_rate: number;
        channel_count: number;
        sample_size: number;
    }

    interface MP4AudioTrack extends MP4MediaTrack
    {
        audio: MP4AudioData;
    }

    type MP4Track = MP4VideoTrack | MP4AudioTrack;

    export interface MP4Info
    {
        duration: number;
        timescale: number;
        fragment_duration: number;
        isFragmented: boolean;
        isProgressive: boolean;
        hasIOD: boolean;
        brands: string[];
        created: Date;
        modified: Date;
        tracks: MP4Track[];
        audioTracks: MP4AudioTrack[];
    }

    interface MP4Sample
    {
        alreadyRead: number;
        chunk_index: number;
        chunk_run_index: number;
        cts: number;
        data: Uint8Array;
        degradation_priority: number;
        depends_on: number;
        description: any;
        description_index: number;
        dts: number;
        duration: number;
        has_redundancy: number;
        is_depended_on: number;
        is_leading: number;
        is_sync: boolean;
        number: number;
        offset: number;
        size: number;
        timescale: number;
        track_id: number;
    }

    export type MP4ArrayBuffer = ArrayBuffer & { fileStart: number };

    export interface MP4File
    {
        onMoovStart?: () => void;
        onReady?: (info: MP4Info) => void;
        onError?: (e: string) => void;
        onSamples?: (id: number, user: any, samples: MP4Sample[]) => any;

        appendBuffer(data: MP4ArrayBuffer): number;
        start(): void;
        stop(): void;
        flush(): void;
        releaseUsedSamples(trackId: number, sampleNumber: number): void;
        setExtractionOptions(trackId: number, user: any, options: { nbSamples?: number, rapAlignment?: number }): void;
    }

    export function createFile(): MP4File;

    export { };
}

[drewlyton]

👋 Hey @JohnWeisz, do you have a further updated version of the types file? I'm looking for the typings for things like DataStream - thanks!

[difosfor]

I've modified and extended the types a bit to be able to use them for WebCodecs VideoDecoder usage:

declare module "mp4box" {

  export interface MP4MediaTrack {
    id: number;
    created: Date;
    modified: Date;
    movie_duration: number;
    movie_timescale: number;
    layer: number;
    alternate_group: number;
    volume: number;
    track_width: number;
    track_height: number;
    timescale: number;
    duration: number;
    bitrate: number;
    codec: string;
    language: string;
    nb_samples: number;
  }

  export interface MP4VideoData {
    width: number;
    height: number;
  }

  export interface MP4VideoTrack extends MP4MediaTrack {
    video: MP4VideoData;
  }

  export interface MP4AudioData {
    sample_rate: number;
    channel_count: number;
    sample_size: number;
  }

  export interface MP4AudioTrack extends MP4MediaTrack {
    audio: MP4AudioData;
  }

  export type MP4Track = MP4VideoTrack | MP4AudioTrack;

  export interface MP4Info {
    duration: number;
    timescale: number;
    fragment_duration: number;
    isFragmented: boolean;
    isProgressive: boolean;
    hasIOD: boolean;
    brands: string[];
    created: Date;
    modified: Date;
    tracks: MP4Track[];
    audioTracks: MP4AudioTrack[];
    videoTracks: MP4VideoTrack[];
  }

  export interface MP4Sample {
    alreadyRead: number;
    chunk_index: number;
    chunk_run_index: number;
    cts: number;
    data: Uint8Array;
    degradation_priority: number;
    depends_on: number;
    description: any;
    description_index: number;
    dts: number;
    duration: number;
    has_redundancy: number;
    is_depended_on: number;
    is_leading: number;
    is_sync: boolean;
    number: number;
    offset: number;
    size: number;
    timescale: number;
    track_id: number;
  }

  export type MP4ArrayBuffer = ArrayBuffer & { fileStart: number };

  export class DataStream {
    static BIG_ENDIAN: boolean;
    static LITTLE_ENDIAN: boolean;
    buffer: ArrayBuffer;
    constructor(arrayBuffer?: ArrayBuffer, byteOffset: number, endianness: boolean): void;
    // TODO: Complete interface
  }

  export interface Trak {
    mdia?: {
      minf?: {
        stbl?: {
          stsd?: {
            entries: {
              avcC?: {
                write: (stream: DataStream) => void
              }
              hvcC?: {
                write: (stream: DataStream) => void
              }
            }[]
          }
        }
      }
    }
    // TODO: Complete interface
  }

  export interface MP4File {
    onMoovStart?: () => void;
    onReady?: (info: MP4Info) => void;
    onError?: (e: string) => void;
    onSamples?: (id: number, user: any, samples: MP4Sample[]) => any;

    appendBuffer(data: MP4ArrayBuffer): number;
    start(): void;
    stop(): void;
    flush(): void;
    releaseUsedSamples(trackId: number, sampleNumber: number): void;
    setExtractionOptions(trackId: number, user?: any, options?: { nbSamples?: number, rapAlignment?: number }): void;
    getTrackById(trackId: number): Trak;
  }

  export function createFile(): MP4File;

  export { };
}

[kixelated]

partially complete types

A few fields are any because I used dts-gen to create the skeleton but didn't manually fix everything.

[wangrongding]

Waiting for best practice. 👀

[difosfor]

I've started to write our own basic parser in strict TS to replace mp4box based on: https://github.com/goldvideo/demuxer/blob/main/src/mp4/demux.ts

[wangrongding]

@difosfor Thanks, but when I npm i demux, it still fails because demux's package.json does not specify a type file, and there is no corresponding type file in the dist folder, which prevents me from importing it directly in ts (which will cause an error).

[difosfor]

@wangrongding I'm not using that package itself, just its mp4 parser code for inspiration. It's MIT licensed. Perhaps we'll open source our demuxer as well later.

[wangrongding]

@difosfor I'm looking forward to your source code, but there doesn't seem to be a good solution at present. 🥲