Support for Canon CR3 raw files

[cytrinox]

This PR adds support for Canon CR3 raw files.

• CRX Decompressor routines are ported from libraw (CrxDecompressor)
• A new basic ISOBMFF parser is added
• A new extended ISOBMFF parser for Canon specific BMFF boxes is added (as part of the Cr3Decoder)
• New cameras.xml definitions for Canon EOS R and EOS M50 added

Canon CR3 files are something new compared to the good old CR2 format. CR2 was based on TIFF. CR3 is based on ISO BMFF. BMFF organizes data into boxes. A description of all related boxes can be found at https://github.com/lclevy/canon_cr3.

To support CR3, rawspeed needs an ISOBMFF parser to iterate and query boxes. This is implemented in IsoMBoxes.cpp. ISOBMFF can be extended with custom boxes, so for the custom Canon boxes, a separate parser is required. The most interesting boxes are:

• CMP1: Box for all parameters needed for decoding/decompression (like encoding type, CFA, compression...)
• IAD1: Box for sensor information (size, crop, black areas)
• CTMD: Box for Canon Timed MetaData, contains the CANONCOLORDATA EXIF tag for white balance coeffs (record 8)

Older cameras has multiple raw quality modes: raw, mraw, sraw with reduced pixel size (=smaller files). Cameras with CR3 support don't shrink the sensor data, instead a wavelet based compression is applied. So pixel dimensions are equal for RAW and CRAW (compact RAW) mode. It makes no sense to define craw as separate mode in cameras.xml as this is directly handled by the decompressor.

But Canon has introduced another mode: 1.6x crop. Because full-frame cameras like R and R5 supports old EF-s lenses, the camera auto-forces a crop-mode with EF-s. For EF and RF lenses, crop-mode can be manually enabled. If crop is enabled, the sensor size, raw pixel data, crop-areas and black-areas are different - compared to full-frame images. This is dynamically handled by the IAD1 information. So crop and black areas are automatically applied and must not be specified in cameras.xml.

To get white balance coeffs is a litte bit tricky. As for CR2 files, Canon stores this information in the CANONCOLORDATA EXIF tag. But this tag is not embedded in CMT1, CMT2, CMT3 or CMT4. Instead, this tag is located in CTMD in track 4. CTMD contains multiple records with a specific type. Record type 8 contains EXIF data and the CANONCOLORDATA tag. The wb_offset hint in cameras.xml points to the wb coeffs inside CANONCOLORDATA. This offsets are taken from exiftool source-code and validated against sample images.

As I have only access to samples for EOS R and EOS M50 yet, these are the first cameras with full support. B/W levels are extracted from large image sets of all ISO values.

So far, I was able to open any CR3 with this code: RAW, CRAW, 1.6-cropped, even dual-pixel images.

I hope this PR helps in adding long awaited CR3 support :smiley:

Fix for #121 #193 #192

[matt-maguire]

So, I was able to open a couple of CR3 files from my Canon M50 using darktable on my intel macbook, with a couple of small issues:

• because I bought it in Japan, it identifies itself as Canon Kiss M, which is a local brandname of the M50 for the Japanese market. By adding a new entry to the cameras.xml file, copied off the M50 entry, then darktable was able to open the file.
• darktable couldn't find the standard color matrix, so I created an entry for the M50 in adobe_coeff.c (it turns out that according to Adobe, the coefficients for M50 should be identical to the M100 coefficients).
• another issue of the darktable side, the build scripts couldn't properly detect on my mac that I'd compiling in ISOBMFF support in my exiv2 library, so that's something to follow up with darktable team.

great job @cytrinox , it appears to be working very well!

[cytrinox]

@matt-maguire

because I bought it in Japan, it identifies itself as Canon Kiss M, which is a local brandname of the M50 for the Japanese market. By adding a new entry to the cameras.xml file, copied off the M50 entry, then darktable was able to open the file.

Please share a patch with me to add the camera entry.

darktable couldn't find the standard color matrix, so I created an entry for the M50 in adobe_coeff.c (it turns out that according to Adobe, the coefficients for M50 should be identical to the M100 coefficients).

There is already a PR for darktable with M50 coeffs from me: https://github.com/darktable-org/darktable/pull/8927 Must there a second entry for the "Kiss M" or is just M50 for your model?

another issue of the darktable side, the build scripts couldn't properly detect on my mac that I'd compiling in ISOBMFF support in my exiv2 library, so that's something to follow up with darktable team.

The darktable PR for ISOBMFF detection was written by myself, too :) https://github.com/darktable-org/darktable/pull/8751 Please let me know what must be fixed or open a PR :smile:

[cytrinox]

I've got a full Canon R5 sample set today, so I will integrate R5 as well.

@LebedevRI I know you want samples at raw.pixls.us. To create a full cameras.xml entry with your dngmeta.rb script, for all ISO levels a sample file is required. So I've requested a sample matrix of:

• Quality mode
  • RAW
  • CRAW
• ISO (L (50), 100, 125, 160, ... HI1, Hi2 (or whatever ISO the camera can reach) in 1/3 ISO levels
• Dualpixel (if camera support it)
  • enabled
  • disabled
• 1.6 Crop mode
  • enabled
  • disabled

All files are named with a good name, like Canon_EOS_R_ISO_100_dualpixel_off_nocrop.CR3. I generate the camera entries with dngmeta.rb after converting each of the sample to dng with Adobe DNG Converter. So I've lot of data (must be ~10 GiB now). Do you want all these samples at raw.pixls.us? Then I need a bulk upload :smile: If you just want a sample for each mode but without ISO variations, I can upload them (if not already a sample is online). I keep the full sets on my disk if nobody wants the data.

[matt-maguire]

Here is a rawspeed cameras.xml patch for the Canon Kiss M camera.

cameras_xml.txt

[matt-maguire]

Rather than create a separate entry for Canon EOS KISS M in cameras.xml, I have added an alias under the M50 entry (since the KISS M is just the Japanese marketing name for the M50 in Japan). This allows the files for the KISS M to be opened, and the id used for the alias allows lensfun to find the appropriate camera profile for lens correction.

I created a PR against the canon_cr3 branch in @cytrinox github repo ( cytrinox/rawspeed#1 ), if he accepts that PR onto his canon_cr3 branch then I think the associated commit should be automatically picked up by this PR.

[cytrinox]

@matt-maguire I've already reworked the cameras.xml for the Kiss M and will push the changes soon.

[cytrinox]

Base samples are uploaded to RPU for all cameras included in this PR. If more/other samples needed, please post an update on this PR and @johnny-bit and myself organize them.

For 90D and 850D, the PR https://github.com/darktable-org/darktable/pull/8962 includes information for adobe_coeff.c.

[tpinfold]

Thanks for all your hard work on this issue.

[Thanatomanic]

Don't hesitate to look for adobe coefficients or black/white levels in our camconst.json file from RawTherapee. It's may help fill in some gaps!

[lclevy]

Please do you have cr3 samples with crop (1.6x with EF-S lens) ? It is to improve my cr3 docs and parser . Thanks

[cytrinox]

Please do you have cr3 samples with crop (1.6x with EF-S lens) ? It is to improve my cr3 docs and parser . Thanks

I've crop samples for all cameras+modes (that supports crop) listed in this PR. Can you contact me at matrix/irc?

[lclevy]

Please do you have cr3 samples with crop (1.6x with EF-S lens) ? It is to improve my cr3 docs and parser . Thanks

I've crop samples for all cameras+modes (that supports crop) listed in this PR. Can you contact me at matrix/irc?

Which address? Server? Or room?

[cytrinox]

Please do you have cr3 samples with crop (1.6x with EF-S lens) ? It is to improve my cr3 docs and parser . Thanks

I've crop samples for all cameras+modes (that supports crop) listed in this PR. Can you contact me at matrix/irc?

Which address? Server? Or room?

#pixls.us or #darktable on freenode @cytrinox

[johnny-bit]

@lclevy - both darktable and pixls.us channels are on freenode irc on martix itself there's #darktable-management:matrix.org which is usually quiet and only for dt management but can be used as a hub for reaching darktable devs on matrix also on matrix there's #exiv2-chat:matrix.org with the devs of exiv2 :)

also the call for samples is present (with links still viable and current sample status) here: https://discuss.pixls.us/t/call-for-extended-cr3-samples-all-iso-combinations-needed/24959?u=johnny-bit

AFAIK cytrinox has all the files sent by users.

[lclevy]

"CRAW (compressed RAW)" craw does not mean compressed raw, but "compact raw" raw (lossless) and craw (lossy) are both compressed

[cytrinox]

Added credit for CR3 file structure info from lclevy/canon_cr3, no other code changes are made.

[cytrinox]

I've added two more commits for fixing ISOSPEEDRATINGS integer overflow and better IAD1 box parsing.

The IAD1 box contains 4 rectangles, one for crop, two for black areas and one for active area. During tests a user reported that for the EOS RP model, black levels (from rawspeed calculation) are wrong for photos using larger ISO values (>= 800). For such images, the black areas include some white pixels which results in wrong black levels.

After some investigation, I think this is a camera bug, as the IAD1 box says blacklevel areas start at offset 0 (for vertial and horizontal) but in fact, there are white pixels located at this offset. I've written a workaround by shifting the black area offset by 12 pixels. This works well on RP and other tested models. If requested, I can enable this workaround only for the EOS RP model but I think the shift is safe for all models.

[codecov[bot]]

Codecov Report

Merging #271 (09ca1df) into develop (0a2d897) will decrease coverage by 9.97%. The diff coverage is 0.04%.

@@             Coverage Diff             @@
##           develop     #271      +/-   ##
===========================================
- Coverage    60.47%   50.50%   -9.98%     
===========================================
  Files          207      214       +7     
  Lines        12628    15121    +2493     
  Branches      1761     2259     +498     
===========================================
  Hits          7637     7637              
- Misses        4859     7352    +2493     
  Partials       132      132              

Flag	Coverage Δ
integration	39.66% <0.04%> (-8.28%)	:arrow_down:
rpu_u	39.66% <0.04%> (-8.28%)	:arrow_down:
unittests	18.83% <0.00%> (-3.74%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
fuzz/librawspeed/decoders/TiffDecoders/main.cpp	6.66% <ø> (ø)
fuzz/librawspeed/parsers/main.cpp	0.00% <ø> (ø)
src/librawspeed/common/Range.h	77.27% <0.00%> (-7.73%)	:arrow_down:
src/librawspeed/decoders/Cr3Decoder.cpp	0.00% <0.00%> (ø)
src/librawspeed/decoders/Cr3Decoder.h	0.00% <0.00%> (ø)
src/librawspeed/decompressors/CrxDecompressor.cpp	0.00% <0.00%> (ø)
src/librawspeed/io/ByteStream.h	75.78% <ø> (ø)
src/librawspeed/io/Endianness.h	87.50% <0.00%> (-4.61%)	:arrow_down:
src/librawspeed/parsers/IsoMParser.cpp	0.00% <0.00%> (ø)
src/librawspeed/parsers/IsoMParserException.h	0.00% <0.00%> (ø)
... and 4 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 0a2d897...09ca1df. Read the comment docs.

[BodoHobbit]

Hi, I really appreciate your work for enabling CR3 support on darktable (especially since my old Canon camera is broken and now I switched to an EOS R). Is there still a chance to get CR3 support in the upcoming darktable release or is it already too late? Is ther anything I can help with? BR, Bodo

[johnny-bit]

the 3.6 release is already past the feature freeze. You can rely on experimental builds and tests if you're ok with that.

[cytrinox]

Rebased to current develop branch and added all missing cameras. The list of cameras is now complete. Each CR3 camera is in the list. For testers: You need this patch, too: https://github.com/darktable-org/darktable/pull/9641

My work on this PR is now finished. For those are not able (or willing) to build experimental code, you can have a look at my other project: https://github.com/dnglab/dnglab It's a FOSS DNG converter for Linux with support for CR3 files. The CR3/Crx decoder was rewritten and documented, but only RAW (not CRAW) files are supported. Edit: RAW, CRAW, PQ-HDR and roll files are supported now.

[silb]

For my PowerShot G5 X Mark II firmware version 1.1.0 I had to make the following modification to this PR to be able to view the CR3 images in raw and craw mode:

diff --git a/src/librawspeed/decoders/Cr3Decoder.cpp b/src/librawspeed/decoders/Cr3Decoder.cpp
index 402debf8..6a1f6ba1 100644
--- a/src/librawspeed/decoders/Cr3Decoder.cpp
+++ b/src/librawspeed/decoders/Cr3Decoder.cpp
@@ -577,6 +577,8 @@ bool Cr3Decoder::isCodecSupported(const std::string& compressorVersion) const {
                                               // (raw)
          || compressorVersion ==
                 "CanonCR3_002/00.10.00/00.00.00" // CR3 of 1DX Mark III (craw)
+         || compressorVersion ==
+                "CanonCR3_001/01.09.00/00.00.00" // PowerShot G5 X Mark II
          || compressorVersion ==
                 "CanonCR3_001/00.09.00/00.00.00"; // EOS R, EOS RP, M50, 250D,
                                                   // 90D, M6 Mark II, M200

Edit: spellings

[LibRaw]

For my PowerShot G5 X Mark II firmware version 1.1.0 I had to make the following modification to this PR to be able to view the CR3 images in raw and craw mode:

@silb , could you please share some sample file(s)?

[silb]

@LibRaw you can download the ISO samples here: http://buggy.secnod.net/~stigi/canon_powershot_g5_x_mark_ii_raws.zip

It is the same set of images I supplied to the extended CR3 samples.

Edit: typo

[LibRaw]

@silb thank you for the samples. LibRaw (library) is not affected by the issue (because of less strict rules used when selecting CR3 track)

[cytrinox]

I've added more strict checks to detect changes in the crx codec very early.

[cytrinox]

@silb Will update the PR soon. Can you have a look at https://discuss.pixls.us/t/call-for-samples-of-canons-raw-burst-shots/26219/1 please? Your G5 cam supports burst film rolls and I need a set of samples. Not for all ISOs but a set of CRAW and RAW with ISO 100 and something high like ISO 6400 (or maximum possible) would be great.

[silb]

@cytrinox see https://discuss.pixls.us/t/call-for-samples-of-canons-raw-burst-shots/26219/2?u=silb for raw burst mode samples.

Thank you for your work on CR3 support.

[paolobenve]

This PR yet shows no previews for CR3 files in import dialog, as I reported in https://github.com/darktable-org/darktable/issues/9935

[cytrinox]

Previews are not related to rawspeed and/or this PR. The preview in the import dialog in darktable uses (if possible) the thumbnail embedded in the RAW file and this is handled by exiv2 only. I recommend to check the exiv2 code if thumbnail extraction for CR3 is already implemented but afaik it's not.