FCP XML adapter: AttributeError: 'NoneType' object has no attribute find / split

[mm3509]

Bug Report

Incorrect Functionality and General Questions

I have video projects that I want to convert from Final Cut Pro to KdenLive. I found the OpenTimelineIO project and it would solve all my problems. I installed with

$ python3 -m pip install opentimelineio
...
$ $ python3 -m pip show opentimelineio
Name: OpenTimelineIO
Version: 0.15.0

I tried the sample code provided:

import opentimelineio as otio

timeline = otio.adapters.read_from_file("/path/to/file.fcpxml")
for clip in timeline.find_clips():
  print(clip.name, clip.duration())

and get the error:

  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 998, in _format_id_for_clip
    resource = self._compound_clip_by_id(
AttributeError: 'NoneType' object has no attribute 'find'

I asked on StackOverflow and replaced the offending lines:

if resource is None:
    resource = self._compound_clip_by_id(
        clip.get("ref")
    ).find("sequence")

with:

if resource is None:
    resource = self._compound_clip_by_id(
        clip.get("ref")
    )
    if resource is None:
        return default_format
    else:
        resource = resource.find("sequence")

It fixes that error and processes several clips, but then I get a similar error:

  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 1059, in _format_frame_duration
    total, rate = media_format.get("frameDuration").split("/")
AttributeError: 'NoneType' object has no attribute 'split'

To Reproduce

1. Operating System: macOS Big Sur 11.7.9
2. Python version: Python 3.8.9
3. OpenTimelineIO release version or commit hash: 0.15.0
4. Compiler information:

Apple clang version 13.0.0 (clang-1300.0.29.30)
Target: x86_64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Expected Behavior

I expected the Python code to process the FCP XML file without errors.

Logs

For the first error, here is the console log:

$ python3 kdenlive.py
Traceback (most recent call last):
  File "kdenlive.py", line 5, in <module>
    timeline = otio.adapters.read_from_file(os.path.expanduser("/path/to/dir/Info.fcpxml"))
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio/adapters/__init__.py", line 137, in read_from_file
    return adapter.read_from_file(
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio/adapters/adapter.py", line 119, in read_from_file
    result = self._execute_function(
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio/plugins/python_plugin.py", line 153, in _execute_function
    return (getattr(self.module(), func_name)(**kwargs))
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 1147, in read_from_string
    return FcpxXml(input_str).to_otio()
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 774, in to_otio
    return self._from_library()
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 785, in _from_library
    return self._from_event(self.fcpx_xml.find("./library/event"))
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 792, in _from_event
    container.append(self._from_project(project))
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 797, in _from_project
    timeline.tracks = self._squence_to_stack(
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 830, in _squence_to_stack
    composable = self._build_composable(
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 877, in _build_composable
    source_range = self._time_range(
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 1084, in _time_range
    self._format_frame_rate(format_id)
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 1059, in _format_frame_rate
    fd_total, fd_rate = self._format_frame_duration(format_id)
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 1054, in _format_frame_duration
    total, rate = media_format.get("frameDuration").split("/")

For the second error, after monkey-patching around line 998, here is the console log:

$ python3 kdenlive.py               
Traceback (most recent call last):
  File "kdenlive.py", line 5, in <module>
    timeline = otio.adapters.read_from_file("/path/to/dir/Info.fcpxml"))
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio/adapters/__init__.py", line 137, in read_from_file
    return adapter.read_from_file(
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio/adapters/adapter.py", line 119, in read_from_file
    result = self._execute_function(
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio/plugins/python_plugin.py", line 153, in _execute_function
    return (getattr(self.module(), func_name)(**kwargs))
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 1150, in read_from_string
    return FcpxXml(input_str).to_otio()
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 774, in to_otio
    return self._from_library()
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 785, in _from_library
    return self._from_event(self.fcpx_xml.find("./library/event"))
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 792, in _from_event
    container.append(self._from_project(project))
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 797, in _from_project
    timeline.tracks = self._squence_to_stack(
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 830, in _squence_to_stack
    composable = self._build_composable(
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 877, in _build_composable
    source_range = self._time_range(
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 1087, in _time_range
    self._format_frame_rate(format_id)
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 1062, in _format_frame_rate
    fd_total, fd_rate = self._format_frame_duration(format_id)
  File "~/Library/Python/3.8/lib/python/site-packages/opentimelineio_contrib/adapters/fcpx_xml.py", line 1057, in _format_frame_duration
    total, rate = media_format.get("frameDuration").split("/")
AttributeError: 'NoneType' object has no attribute 'split'

Additional Context

You can find here a link to a complete FCP XML file that causes this error in order to reproduce it.

[mm3509]

Thank you @reinecke for handling this. I need to deal with the files from Final Cut Pro in the next couple months. Do you think it likely to have a fix for this issue in that timeframe or should I keep trying to "monkey-patch" the source code to work around these errors?

[reinecke]

I took a run at digging into this issue today - digging into the adapter, it looks as if the adapter has some problematic time math at a pretty deep level.

Some issues I see are:

• Frame rates are often truncated to integer frame rates (1, 2, 3)
• Calculations are often converting to frame counts which can be quite problematic with audio and the conversions are often built in a way where they don't preserve appropriate precision
• There is a lot repetition in the code. For instance, there are duplicate implementations of converting time fraction strings (of the form 10s or 100/25s) into either frames, RationalTime, or time durations. There should be one implementation converting these to python Fraction objects and then those can be inverted to rates when they are a frameDuration or converted to RationalTime and potentially rescaled to a format rate when appropriate.

I don't have a lot of exposure to fcpxml in my pipelines so I don't think I'm the right person to take this work on as I wouldn't be able to validate my results very well. If someone else is interested in taking this on, I'd be happy to share some direction and code about what needs to be done.

Also, of note, any of this work should take place over at the broken out adapter repo.

[reinecke]

To provide more guidance into the type of refactoring that needs to be done, I pushed a branch in my fork of the adapter repo.

I was working as quickly as possible to see if I could address the issue, so I didn't do a detailed analysis to ensure correctness ot the changes I started applying.

I got as far as reaching a new exception with the sample EDL that @miguelmorin provided:

src/otio_fcpx_xml_adapter/fcpx_xml.py:1017: in _offset_and_lane
    parent_format_id = self._format_id_for_clip(parent, default_format)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <opentimelineio.adapters.fcpx_xml.FcpxXml object at 0x105087e90>
clip = <Element 'title' at 0x105159cb0>, default_format = 'r1'

    def _format_id_for_clip(self, clip, default_format):
        if not clip.get("ref", None) or clip.tag == "gap":
            return default_format

        resource = self._asset_by_id(clip.get("ref"))

        if resource is None:
            try:
                resource = self._compound_clip_by_id(
                    clip.get("ref")
                ).find("sequence")
            except AttributeError:
                pass

>       return resource.get("format", default_format)
E       AttributeError: 'NoneType' object has no attribute 'get'

I'm not so familiar with FCP X XML, so I don't really know the correct way to address this. Someone with more FCP X XML experience can maybe pick up my branch and run with it?

[apetrynet]

I believe the original author of this adapter is @eric-with-a-c. Perhaps he can shed some light on this?

I don't know exactly how it happened, but he seems to have fallen out of the contributors list when we transferred the adapter to the new repo. Sorry about that @eric-with-a-c !

[eric-with-a-c]

In addition to the time math issues @reinecke mentions above, the fcpxml adapter only supports fcpxmls up to version 1.8. The current version of fcpxml exported by Final Cut Pro X by default is 1.11, with the option of exporting a 1.10 version. There were some big changes introduced in 1.9 or 1.10. For example, the whole media-rep concept was introduced in 1.9 or 1.10 which seems to have changed the parenting of some elements in the fcpxml. Something like that may be what's causing things to fail even with the adjustments made by @reinecke.

It's probably fair to say, given the time math issues and the old fcpxml version support, the adapter needs a fair bit of work to be usable with a current project. I had started that work a while ago, but never had the time to finish.

[mm3509]

Thank you @reinecke , @apetrynet and @eric-with-a-c for trying to solve this problem. It seems to have very high cost-benefit ratio, given the bugs you already found and the issues from version 1.8 and 1.11, and how few people seem to need this adapter. In the meantime, I'll cut this Gordian knot by running Final Cut Pro, exporting all the assets I need (e.g., a video without titles nor captions, an audio track with only sound effects), and reassembling them inside Kdenlive. I'll keep the source files in case some brave soul in the future is able to fix this, and I can run the tests on my projects (about one hour worth of projects in Final Cut Pro, and another in Adobre Premiere Pro that I can convert via Final Cut Pro).