OpenTimelineIO icon indicating copy to clipboard operation
OpenTimelineIO copied to clipboard

Implement subtitles schema [WIP]

Open KarthikRIyer opened this issue 4 years ago • 7 comments

Link the Issue(s) this Pull Request is related to.

Closes #62

Summarize your change.

This PR adds basic support for Subtitles. I've based it on what I understood from TTML2 and from my experience with Premiere Pro. I've summarized the changes in the issue (#62). Here's an image summarizing the changes (Note change: SerializableObject instead of SerializableObjectWithMetadata):

image

Some concerns:

  • I think storing a Style object with each TimedText is wasteful, but decided to have this approach because we don't have id system in otio yet. In TTML2 there's a section to specify all styles and then refer then in each TimedText. With the current approach conversion from otio to say TTML2 won't be so direct because we don't have a single set of styles. One other thing I thought of was to have a list of styles inside Subtitle and then have a string id in each TimedText. But then each TimedText object will have no style meaning independently/outside the Subtitle. Thoughts?

  • What are Markers used for? Each marker has a markedRange. When we trim an Item or change the sourceRange shouldn't the markedRange of the markers at the start or end of the Item also change? I couldn't find anything that does something like this. Or is this handled in another way? Why I'm asking is if we trim the Subtitle or change its sourceRange, the ranges of TimedTexts at the start or end might change.

Reference associated tests.

Will be added later.

KarthikRIyer avatar Sep 24 '20 20:09 KarthikRIyer

Similar to ImageSequenceReference, I wonder if implementing this in an adapter to try out the ergonomics might be a good idea? Also, it might be useful to see some examples of use cases and the code it takes to achieve them using the schema. In ImageSequenceReference, @apetrynet did a lot of iterations trying it out to make sure it worked like he expected and that really helped us make sure we had a usable schema.

reinecke avatar Oct 01 '20 18:10 reinecke

A usage example could be to write an adapter that can use the Subtitle Schema and write out a sub format that ffmpeg supports. I don't think ffmpeg supports TTML2 so converting to srt or something with an adapter might be a good test.

apetrynet avatar Oct 01 '20 18:10 apetrynet

Yes, I was thinking of an srt adapter. Was waiting to hear others' thoughts on the parameters I've included and if we need to implement any other functionality.

I'll write a simple srt adapter and get back to you.

KarthikRIyer avatar Oct 01 '20 18:10 KarthikRIyer

@apetrynet @reinecke I've implemented an srt adapter.

The tests pass for python3, but there are some extra carriage returns added to the strings with python2. Any idea why this is happening?

KarthikRIyer avatar Oct 20 '20 20:10 KarthikRIyer

Codecov Report

Merging #805 into master will decrease coverage by 1.60%. The diff coverage is 54.85%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #805      +/-   ##
==========================================
- Coverage   84.02%   82.42%   -1.61%     
==========================================
  Files          74       80       +6     
  Lines        3124     3266     +142     
==========================================
+ Hits         2625     2692      +67     
- Misses        499      574      +75     
Flag Coverage Δ
#py27 ?
#py37 ?
#py38 ?
#unittests 82.42% <54.85%> (-1.76%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/opentimelineio/timedTextStyle.h 0.00% <0.00%> (ø)
src/opentimelineio/subtitles.h 3.44% <3.44%> (ø)
src/opentimelineio/subtitles.cpp 38.46% <38.46%> (ø)
src/opentimelineio/timedText.h 46.15% <46.15%> (ø)
src/opentimelineio/timedTextStyle.cpp 58.33% <58.33%> (ø)
src/opentimelineio/timedText.cpp 66.66% <66.66%> (ø)
src/opentimelineio/typeRegistry.cpp 85.29% <100.00%> (ø)
...entimelineio-bindings/otio_serializableObjects.cpp 95.82% <100.00%> (+0.40%) :arrow_up:
...ntimelineio/opentimelineio-bindings/otio_utils.cpp 86.90% <0.00%> (-0.46%) :arrow_down:
... and 25 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 22e2468...9a0665b. Read the comment docs.

codecov-io avatar Oct 21 '20 08:10 codecov-io

I changed os.open() to io.open() and it worked. But I have no idea why :/

KarthikRIyer avatar Oct 21 '20 08:10 KarthikRIyer

Timeline:
    Track1: [----Subtitles----]

If I have something like this and I do: Timeline.children_if(descended_from=otio.schema.Subtitles) This returns [Track1, Subtitles]. Is this the intended behaviour? Shouldn't it return just Subtitles?

@darbyjohnston @meshula

KarthikRIyer avatar Aug 05 '21 03:08 KarthikRIyer