Frozen when rendering a clip in a playlist in a playlist

[LeifAndersen]

I am trying to make a video where I have a clip in a playlist in a playlsit. Which is to say, The main producer is a playlist with one producer, which is another playlist with one producer, which is a m4v video clip.

I output the result to an xml file called out.xml, and then use the melt command line application to run it:

melt out.xml

When I do this the clip plays the first frame and then freezes. Additionally, when I add other clips to the inner playlist only the last clip is shown. Is this a bug in the XML producer or consumer, or am I missing something here?

For reference, the following is the xml file:

<?xml version="1.0" encoding="utf-8"?>
<mlt LC_NUMERIC="en_US.UTF-8" version="6.2.0" root="/Users/leif/src/video/private" title="Anonymous Submission" producer="playlist0">
  <profile description="DV/DVD PAL" width="720" height="576" progressive="0" sample_aspect_num="16" sample_aspect_den="15" display_aspect_num="4" display_aspect_den="3" frame_rate_num="25" frame_rate_den="1" colorspace="601"/>
  <producer id="producer0" in="0" out="5001">
    <property name="length">5002</property>
    <property name="eof">pause</property>
    <property name="resource">/Users/leif/demo.mkv</property>
    <property name="meta.media.nb_streams">2</property>
    <property name="meta.media.0.stream.type">video</property>
    <property name="meta.media.0.stream.frame_rate">30</property>
    <property name="meta.media.0.stream.sample_aspect_ratio">1</property>
    <property name="meta.media.0.codec.width">1920</property>
    <property name="meta.media.0.codec.height">1080</property>
    <property name="meta.media.0.codec.rotate">0</property>
    <property name="meta.media.0.codec.frame_rate">1000</property>
    <property name="meta.media.0.codec.pix_fmt">yuv420p</property>
    <property name="meta.media.0.codec.sample_aspect_ratio">0</property>
    <property name="meta.media.0.codec.colorspace">709</property>
    <property name="meta.media.0.codec.name">vp9</property>
    <property name="meta.media.0.codec.long_name">Google VP9</property>
    <property name="meta.media.0.codec.bit_rate">0</property>
    <property name="meta.attr.0.stream.LANGUAGE.markup">eng</property>
    <property name="meta.attr.0.stream.DURATION.markup">00:03:20.000000000</property>
    <property name="meta.media.1.stream.type">audio</property>
    <property name="meta.media.1.codec.sample_fmt">fltp</property>
    <property name="meta.media.1.codec.sample_rate">44100</property>
    <property name="meta.media.1.codec.channels">2</property>
    <property name="meta.media.1.codec.name">aac</property>
    <property name="meta.media.1.codec.long_name">AAC (Advanced Audio Coding)</property>
    <property name="meta.media.1.codec.bit_rate">0</property>
    <property name="meta.attr.1.stream.CREATION_TIME.markup">2015-05-18 08:29:00</property>
    <property name="meta.attr.1.stream.HANDLER_NAME.markup">SoundHandler</property>
    <property name="meta.attr.1.stream.DURATION.markup">00:03:20.086000000</property>
    <property name="meta.attr.ENCODER.markup">Lavf56.40.101</property>
    <property name="seekable">1</property>
    <property name="meta.media.sample_aspect_num">1</property>
    <property name="meta.media.sample_aspect_den">1</property>
    <property name="aspect_ratio">1</property>
    <property name="audio_index">1</property>
    <property name="video_index">0</property>
    <property name="mute_on_pause">1</property>
    <property name="mlt_service">avformat</property>
  </producer>
  <playlist id="playlist1">
    <entry producer="producer0" in="0" out="5001"/>
  </playlist>
  <playlist id="playlist0" title="Anonymous Submission">
    <entry producer="playlist1" in="" out=""/>
  </playlist>
</mlt>

Any suggestions? Thank you.

[LeifAndersen]

Also, when I manually modify the playlist0 file to have the same start and end times as playlist1:

  <playlist id="playlist0" title="Anonymous Submission">
    <entry producer="playlist1" in="0" out="5001"/>
  </playlist>

The video plays. Which makes me wonder why the xml generator didn't put that there in the first place.

[LeifAndersen]

I should also mention that I am explicitly setting the "out" property to 5001 on the playlists. But they seem to be discarded by the XML generator.

[ddennedy]

I highly doubt anyone will help you with this, and I do not have time to investigate it as a bug any time soon. I suggest you tinker around to figure it yourself and strongly consider not trying to nest playlists. Most tested/used cases of nesting things includes usage of the tractor element. Good luck.

[LeifAndersen]

Sigh, well thanks for the honest answer. I'll see if I can get it to work nesting with tractors. Thanks.

[LeifAndersen]

Also, interesting, you get this same issue when you put a tractor in a playlist:

...
  <tractor id="tractor0" in="" out="">
    <track producer="producer0"/>
  </tractor>
  <playlist id="playlist0" title="Anonymous Submission">
    <entry producer="tractor0" in="" out=""/>
  </playlist>
...

[LeifAndersen]

Hmm...ya, pretty sure at this point that this is a bug. This issue even persists when I use mlt_playlist_append_io.

@ddennedy Do you know where the code for the XML renderer is?

[ddennedy]

In general, I recommend people to stick to composition patterns that one can observe in the XML outputs from melt -consumer xml, Shotcut, and Kdenlive. SImply reading the documentation and trying things can lead to a lot of frustration. However, in this case, it does seem one ought to be able to use any producer as a playlist entry including a playlist. One could test this theory by making a short program or script. To answer your question see src/modules/xml/producer_xml.c

[LeifAndersen]

In general, I recommend people to stick to composition patterns that one can observe in the XML outputs from melt -consumer xml, Shotcut, and Kdenlive

Oh interesting, doesn't that mean everything needs to fall into just one tractor with flat playlists?

(I say that because as far as I can tell, you only really offset a clip in a track is to make it a playlist and insert a blank. Let me know if that is not correct.)

SImply reading the documentation and trying things can lead to a lot of frustration

Ya, I noticed. ;) If you have a better way though I'd love to hear about it. Thank you for all of the help you've given me so far though.

However, in this case, it does seem one ought to be able to use any producer as a playlist entry including a playlist

Yup. The idea being that abstraction is nice. Namely, my motivation for playing with these bindings is because I am working on a Racket based DSL for building videos, and I'm currently playing with using libmlt as a backend. If you are interested in checking it out sometime, the repo is at:

https://github.com/Leifandersen/video

One could test this theory by making a short program or script

Yup, done that. ;) That's how I generated the XML file.

To answer your question see src/modules/xml/producer_xml.c

Cool, thanks. Ya, this looks like a this is a bug in the xml generator. I'll play with a bit and submit a PR.

Thanks again, you're fantastic. :)

[ddennedy]

Oh interesting, doesn't that mean everything needs to fall into just one tractor with flat playlists?

Not necessarily. Shotcut lets you make clip, playlist, or multitrack projects by simply not using the respective features/panels. In a Shotcut multitrack timeline project, when you make a transition you can see an example of a nested tractor (a tractor implies a multitrack - not simply a wrapper for any producer). Also, if you use both playlist and timeline, you will see an orphan playlist. Compositions can have multiple top-level, unreferenced producers, but the xml producer returns the last-most, top-level producer to consume by default. Surely there are other composition patterns that will work but may only be discovered through heavy experimentation alone or experimentation combined with code analysis.