Superfluous files in DCPs and side effects

[wolfgangw]

Checking for superfluous files in DCPs would produce useful hints.

Today, Oct 1st 2012, Mattias Mattsson added a great report to the mix which changes the picture substantially:

A package with superfluous subtitle files (allegedly, I need to see AssetMap and PKL) which makes a Dolby DSS200/Cat 862/TI projector subs combo choke and disable subtitles on playback. As of now this flies completely under dcp_inspect's radar.

Awesome field-report, Mattias, as usual. Thanks.

More coming up ...

[matmat]

Note that Dolbys software completely ignores ASSETMAP, VOLINDEX and PKL files on ingest. Every asset is found by scanning the media, caching the uuids and then looking in the CPL for the ones that are needed. I am a bit confused as to how it manages to handle ttf:s for DCSubtitles with this scheme.. (excactly this process has had som bugs..).

Here is an excerpt from an ftp ingest on a Dolby system, where the software deletes fully compliant PKL, ASSETMAP and VOLINDEX files.

2014-12-28 21:03:32,783 INFO [Thread-11737] content - The file: /var/showstore/content/JAG-STANNAR-TIDEN_FTR-1_F_SV-XX_51_2K_20141107_OV/VOLINDEX uploaded to the FTP server was not recognized as Digital Cinema content and was deleted 2014-12-28 21:06:48,578 INFO [Thread-11738] content - The file: /var/showstore/content/JAG-STANNAR-TIDEN_FTR-1_F_SV-XX_51_2K_20141107_OV/ASSETMAP uploaded to the FTP server was not recognized as Digital Cinema content and was deleted 2014-12-28 21:26:59,004 INFO [Thread-11738] content - The file: /var/showstore/content/JAG-STANNAR-TIDEN_FTR-1_F_SV-XX_51_2K_20141107_OV/PKL_cabac121-30b0-4630-81c4-415e28d3d968.xml uploaded to the FTP server was not recognized as Digital Cinema content and was deleted 2014-12-28 21:28:12,578 INFO [Thread-11738] content - The file: /var/showstore/content/JAG-STANNAR-TIDEN_FTR-1_F_SV-EN_51_2K_20141117_VF-1/PKL_0665d66e-4d36-45bd-9402-30ac44f176f4.xml uploaded to the FTP server was not recognized as Digital Cinema content and was deleted 2014-12-28 21:28:12,768 INFO [Thread-11738] content - The file: /var/showstore/content/JAG-STANNAR-TIDEN_FTR-1_F_SV-EN_51_2K_20141117_VF-1/VOLINDEX uploaded to the FTP server was not recognized as Digital Cinema content and was deleted 2014-12-28 21:28:15,865 INFO [Thread-11738] content - The file: /var/showstore/content/JAG-STANNAR-TIDEN_FTR-1_F_SV-EN_51_2K_20141117_VF-1/ASSETMAP uploaded to the FTP server was not recognized as Digital Cinema content and was deleted

[wolfgangw]

Yes, they are nuts.

The Dolby issue at the time was triggered by a package which, for some reason, had multiple identical DCSubtitle files in each of the subtitle folders respectively. They were named as such:

BourneLegacy_Swedish_R1.xml
BourneLegacy_Swedish_R1 (2).xml
BourneLegacy_Swedish_R1 (3).xml
[...]

Someone had gone overboard with making sure the files were there. The Dolby system had, for some reason, chosen the 2nd (BourneLegacy_Swedish_R1 (2).xml) and then choked and died -- quite likely because of the whitespace in the filename.

I thought it was beautiful :)

Apart from the fact that Dolby needs to get their act together (none of their systems are actually DCI-compliant because of this alone): dcp_inspect still has no mechanism to warn about this.

[matmat]

Actually, the ingest in this case went fine. When there are several files with the same uuid which one is chosen is somewhat undefined (it shouldn't really matter as the content should be the same).

What went wrong in the above example was NOT the ingest stage but really when the Dolby httpd did not escape whitespace in the URI presented to the projector for CineCanvas rendering. This would go wrong even with non-duplicate files containing whitespace. This is now fixed in recent versions.

I can agree that the Dolby way of discarding everything but assets and the CPL (and then generating their own) is somewhat weird. But it is also rather robust. It means for example that the order that a VF package is ingested doesn't matter (before or after the OV) and that you could export a VF as a complete package without the OV version.

All DCinema files have uuid:s so filenames should not really matter as long as the files are found. You have to be a bit more careful with the ttf:s and png:s of course.

[matmat]

Here is the (VF) package in question: http://drive.google.com/file/d/0B3VHpr6OshToZ2hGRy1ZVnRfNGs

As you can see the package is a valid supplemental package. The duplicated files were created by operator error when trying to extract the archive several times to the same location and using the Windows "feature" of renaming the files.

There are downsides to this strategy of Dolbys' of scanning every file on the ingest medium of course. Some versions does not cope well with non-latin characters in file names..

Maybe non-latin file name characters is worthy of a warning in dcp_inspect?

[overlookmotel]

+1 for warning of non-latin characters in file names. I had a DCP recently fail to ingest (don't know what server model I'm afraid) due to a Chinese character in the MXF file names. It passed dcp_inspect's checks with no errors.

Renaming the files and manually editing the XML to match solved the problem.