[Feature Request] Video Embeds

[Stanzilla]

Since proper Markdown support is not going to happen anytime soon, would it be possible to implement it in material-extensions like suggested here? https://talk.commonmark.org/t/embedded-audio-and-video/441/102

There is https://github.com/cmrd-senya/markdown-it-html5-embed as an example implementation.

[facelessuser]

So it looks at extensions?

[Stanzilla]

Well it could, when implementing it like this

Media	Directive	Shorthand
Image	:image{}	{}
Video	:video{}	^{}
Audio	:audio{}	@{}

You could grab the extension name from the filename and use it in the type parameter.

So

:video[Description "Poster.jpg"](test.mp4)

Would end up as

<video autoplay loop muted playsinline poster="Poster.jpg">
    <source src="test.mp4" type="video/mp4" />
</video>

Or you could specify like this

![](video.mp4){type: video/mp4}

[facelessuser]

Why :image{} and {} for image? We already have a syntax for images.

^ would get confused when working with subscripts (or maybe I mean superscripts :shrug:) see pymdownx.caret extension.

I'd have to think about this as right now, I'm not sure on the syntax.

[Stanzilla]

I just picked a random example, really

[facelessuser]

For me, features with a fully fleshed out proposal have a better chance of consideration. Either I have to sit down and come up with it (unless I have a direct need for it, this becomes low priority) or the issue filer should present it. So I took it as a real proposal, one I'm not too keen on.

I could see someone maybe just doing something like this with attr_list.

![](video.mp4){: type="video/mp3"}

Then just use a tree processor to look for images with a type attribute of video/* and transform accordingly.

With that said though, it feels weird using images for this, but someone could throw together a quick and dirty Markdown extension to do this. They could even use this method to embed youtube.

Without a syntax that I'm fully behind, I'm not sure I would do this. I'd have to think about this.

[facelessuser]

I guess if you look at the ![] syntax as more of a media syntax than an image syntax, then maybe it makes sense to do it this way 🤷‍♂️ .

[Stanzilla]

I meant I picked a random suggestion from the topic I linked.

I guess if you look at the ![] syntax as more of a media syntax than an image syntax, then maybe it makes sense to do it this way 🤷‍♂️ .

Yeah that's where ![](video.mp4){type: video/mp4} came from.

https://github.com/cmrd-senya/markdown-it-html5-embed supports two modes as well, the MagicLink one and image syntax one ![](https://example.com/file.webm)

[facelessuser]

markdown-it example is probably more inline with a direction I would consider at this point. it kind of meshes with what we already do. We could easily just scan link extensions and convert them to media elements. I'm curious what you gain from using image format vs link format. I wonder if I would even waste time implementing both forms. Maybe it allows you to more easily identify media links instead of scanning all links 🤷‍♂️ . I'd have to look into it more. Something to consider though.

[facelessuser]

@gir-bot remove S: triage @gir-bot add T: feature, S: maybe, C: magiclink

[facelessuser]

@gir-bot add P: maybe

[kdeldycke]

For my own video needs in Markdown I rely on mdx-video. It worked well so far even if the project has not seen any activity since 2016.

[facelessuser]

mdx-video seems to be related to media service embedding, not video file embeds. As I am going through the backlog again, what video formats, assuming I implement this, do we think are manditory? Before I even begin looking into this, I kind of want to get a feel for all the mimetypes and formats and such.

[Stanzilla]

I'd say mp4 and webm as containers

[facelessuser]

Sounds good, I'm debating looking into at least this one for the next release. I may skip YouTube and such for now.

[facelessuser]

Seemed pretty easy to get something up and going. We don't even really have to do anything special. We just let Magiclink find the links as normal, or the normal link methods, and then we use a treeprocessor right after attr_list to find the links. Copy over all the attributes so you can set width and whatever inline, wrap it all up, and videos:

https://dl8.webmfiles.org/elephants-dream.webm

[Tiny video](./images/elephants-dream.webm){width="50"}

I like this because we don't really need to do anything special, we just let the user drop in the links, and we just handle it.

Let me know what you'd want out of such functionality. Are there things we feel we need to expose options to?

[Stanzilla]

Oh wow, yeah, that is really great. Could you post the generated HTML as well?

[facelessuser]

Yeah, I just referenced the markdown-it plugin's readme and used that format:

<video controls width=50>
<source src=./images/elephants-dream.webm type=video/webm>
Your browser does not support playing HTML5 video. You can<a href=images/elephants-dream.webm>download a copy of the video file</a>instead.
</video>

I guess they also will inject a content description from the title (or maybe the alt -- from somewhere 🤷🏻 ) which we could maybe do as well.

It should map both webm and mp4 as video/<type>. I didn't play with any audio files. I figure maybe we'll just start with this.

[Stanzilla]

Ah yeah I think that would work, as comparision, this is the "standard" of how I do my videos:

<div class="mt-video">
    <video autoplay loop muted playsinline poster="/img/EditorManual/Abilities/Gem.png">
        <source src="/img/MyFirstMultiplayer/DragDropBushes.webm" type="video/webm" />
        <source src="/img/MyFirstMultiplayer/DragDropBushes.mp4" type="video/mp4" />
    </video>
</div>

[facelessuser]

Okay, so it looks like you have fallbacks...I wonder if we could make something like that work somehow...

[facelessuser]

Okay, here's an idea. What if when videos are grouped together the first is taken as the main one, and all the subsequent ones are fallbacks.

So here, the first video is the main one, the second is the fallback, and the third one spawns a new video. I decided that maybe the img syntax is appropriate because then we don't need to deal with people nesting content under the anchor tag.

![Take these attributes](https://dl8.webmfiles.org/elephants-dream.webm){width="300"}
![Don't take the fallback's size](./images/elephants-dream.webm){width="50"}

![A new group](https://dl8.webmfiles.org/elephants-dream.webm){width="300"}

Let me know what you think. This is all kind of an experiment right now.

[facelessuser]

As an alternative, we could link them only if they are marked with a fallback attribute:

![](video.webm)
![](video.mp4){fallback=''}

[facelessuser]

We now have audio and video supported. We assume video for webm, but if a user overrides with an audio type attribute, we will respect it.

We also now sort out attr_list attributes applying <source> related attributes to that element and the rest to the media container (audio or video).

I'll probably scan around for commonly supported mime types and add them.

After that, it is just deciding if we want to have fallback logic, and if so, what does that look like.

Hopefully, this will make it in the 9.0 release, but we'll see. I don't want to rush it if it ends up being half-baked.

[Stanzilla]

I think I like the marking as fallback idea more because it's a little clearer to read.

Would something like this work as well?

![](video.webm){fallback='video.mp4'}

[facelessuser]

Experimental branch here: https://github.com/facelessuser/pymdown-extensions/tree/feature/magic-media

Would something like this work as well?

![](video.webm){fallback='video.mp4'}

Not currently. But how would you chain more than one fallback? Maybe if we require a separator like ;...It'd definitely make parsing things easier, but then you can't set source-specific attributes: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/source#attributes.

[facelessuser]

Just an FYI, I haven't implemented the fallback attribute at all yet. Currently, it just groups clustered videos or audios...

[facelessuser]

Implemented fallback attribute for explicit grouping. Once again provides an absolute last fallback message with download link. alt will be used as a description in the message if provided. The first provided video or audio file in the chain will be the one in the download and will be the one that provides the alt description.

By default, it will assume the following MIME types based on the extension but will accept a MIME type override as mp4, webm, and I believe even ogg can all technically be used as either video or audio.

    # Default MIME types, but can be overridden
    MIMEMAP = {
        'mp4': 'video/mp4',
        'webm': 'video/webm',
        'mp3': 'audio/mpeg',
        'ogg': 'audio/ogg',
        'wav': 'audio/wav',
        'flac': 'audio/flac'
    }

---

TODO: maybe tomorrow

The only other thing I can think to add is maybe some kind of default attribute option. Like if you always set loop to all your videos or audio or whatever, you can maybe define some defaults and they'll automatically be applied, though user set attributes will override them.

[facelessuser]

Okay, everything is up in pull request #1465. Feedback is welcome. Let me know if this is what everyone was hoping for.

Still need to write tests and docs once we've decided if this is a viable direction.

[Stanzilla]

I will test it, I'm not sure when I'll have time but I hope next week!

[facelessuser]

No worries. Take your time.

[facelessuser]

Don't know if I'll get to it, but probably need subtitle support <track> as well. Maye a later issue.