rawspeed icon indicating copy to clipboard operation
rawspeed copied to clipboard

Nikon Z 9 support (lossless only)

Open kmilos opened this issue 3 years ago • 9 comments

Addresses https://github.com/darktable-org/darktable/issues/10889 and https://github.com/darktable-org/darktable/issues/11067

@LebedevRI This is a stub only, I have not verified the CFA pattern nor the crop.

There is also no support (nor detection & exception throwing!) added for the new "High Efficiency" codec.

kmilos avatar Feb 24 '22 14:02 kmilos

Codecov Report

Merging #347 (47713f8) into develop (f4b9a08) will increase coverage by 0.02%. The diff coverage is n/a.

:exclamation: Current head 47713f8 differs from pull request most recent head 2b71bf0. Consider uploading reports for the commit 2b71bf0 to get more accurate results

@@             Coverage Diff             @@
##           develop     #347      +/-   ##
===========================================
+ Coverage    60.45%   60.47%   +0.02%     
===========================================
  Files          207      207              
  Lines        12629    12628       -1     
  Branches      1762     1761       -1     
===========================================
+ Hits          7635     7637       +2     
+ Misses        4862     4859       -3     
  Partials       132      132              
Flag Coverage Δ
integration 47.94% <ø> (+0.03%) :arrow_up:
rpu_u 47.94% <ø> (+0.03%) :arrow_up:
unittests 22.57% <ø> (-0.01%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/librawspeed/metadata/ColorFilterArray.cpp 97.24% <0.00%> (-0.03%) :arrow_down:
src/librawspeed/tiff/CiffIFD.cpp 60.29% <0.00%> (ø)
src/librawspeed/decompressors/HuffmanTableTree.h 0.00% <0.00%> (ø)
src/librawspeed/decompressors/VC5Decompressor.cpp 75.91% <0.00%> (+0.21%) :arrow_up:
src/librawspeed/decoders/DngDecoder.cpp 50.90% <0.00%> (+0.22%) :arrow_up:
src/librawspeed/decoders/NefDecoder.cpp 20.22% <0.00%> (+0.22%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f4b9a08...2b71bf0. Read the comment docs.

codecov[bot] avatar Feb 24 '22 14:02 codecov[bot]

Just a little bit of info I have found re the new HE modes:

They do not have the usual 0x96 NEFLinearizationTable tag in the MakerNote

The fallback 0x8c is junk/unknown, so any further parsing crashes rawspeed

There seems to be a new 0x51 tag instead in Z 9 files, here's the comparison of lossless vs HE exiftool output:

image

Note that the NEFCompression field of that tag seems to correspond to tag 0x93 (which is absent for Z 9).

There is also a new (unknown) 0xc7d5 NEFInfo IFD in the Z bodies as well.

kmilos avatar Feb 24 '22 14:02 kmilos

The RPU lossless sample loads fine in dt master and there are no artifacts at the edges, so this might be good to go w/o the crop (like the Z 7_2).

Loading the HE files luckily does not crash dt (the unsupported message is displayed), so additional checks might not be needed after all.

kmilos avatar Mar 01 '22 14:03 kmilos

This seems to be stuck. Can I help at all (I'm a software engineer by profession)? Looking at the Details of the failing check, it seems to be a python module that is no longer available, but a replacement or two are indicated, so it ought to be easy to fix.

colin-adams avatar Mar 28 '22 12:03 colin-adams

Cleared that hurdle, but a new one appears of course, and I have zero interest in addressing it: CodeChecker (external repository) requires older psutil 5.8.0 exactly and a) wheel is not available for Python 3.10 (psutil 5.9.0 is already installed) so it tries to build 5.8.0 from source, and as a consequence b) setuptools can't figure out it needs to use the LLVM linker, it assumes GCC instead.

In any case, this is not blocking this particular PR, as there is no code change.

kmilos avatar Mar 29 '22 14:03 kmilos

What's holding this up? Anything I can do to help?

colin-adams avatar Apr 08 '22 10:04 colin-adams

So this will add ticoraw for upcoming 2.0 firmware, like in Canon and Red? https://www.nikoncafe.com/threads/nikon-z9-raw-file-support.331229/

https://www.newsshooter.com/2022/04/14/nikon-z9-firmware-2-0-upgrade-will-include-8-3k-60p-12-bit-internal-raw-4-1k-60p-in-prores-raw/ https://youtu.be/BC_oyujpuDM

ValZapod avatar Apr 15 '22 04:04 ValZapod

Any updates? I've verified the settings in the XML with my own Z 9, they're fine.

Sturmflut avatar Jul 01 '22 22:07 Sturmflut

So this will add ticoraw for upcoming 2.0 firmware, like in Canon and Red?

No, this only adds support for the sensor in the Z 9.

Sturmflut avatar Jul 03 '22 14:07 Sturmflut

I don't know if my remark here is relevant, but I have been using the Z9 with darktable since the beginning of August, and I haven't ever seen any superfluous columns of pixels on the right (or anywhere else for that matter).

On Fri, 18 Nov 2022 at 18:50, Roman Lebedev @.***> wrote:

@.**** commented on this pull request.

In data/cameras.xml https://github.com/darktable-org/rawspeed/pull/347#discussion_r1026775105 :

@@ -4379,6 +4379,24 @@ </ColorMatrix> </ColorMatrices> </Camera>

  • <ID make="Nikon" model="Z 9">Nikon Z 9</ID>
    
  • <CFA width="2" height="2">
    
  • 	<Color x="0" y="0">RED</Color>
    
  • 	<Color x="1" y="0">GREEN</Color>
    
  • 	<Color x="0" y="1">GREEN</Color>
    
  • 	<Color x="1" y="1">BLUE</Color>
    
  • </CFA>
    
  • <Crop x="0" y="0" width="0" height="0"/>
    

I can not tell from existing samples, but it looks like this needs a bit of a crop on the right.

— Reply to this email directly, view it on GitHub https://github.com/darktable-org/rawspeed/pull/347#pullrequestreview-1186623328, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA6H5ZEOYLO6SHQXH4J55DWI7FWZANCNFSM5PHPMWVA . You are receiving this because you commented.Message ID: @.***>

colin-adams avatar Nov 18 '22 18:11 colin-adams

It's basically off by at most 6-12 pixels, not really noticeable unless you really look.

LebedevRI avatar Nov 18 '22 18:11 LebedevRI

@kmilos thank you!

LebedevRI avatar Nov 18 '22 19:11 LebedevRI

What does it mean that the nikon_z9 branch was deleted? There is still no official support for Nikon Z9? I've updated my cameras.xml file but my path is /usr/share/darktable/rawspeed/cameras.xml not /usr/share/darktable/rawspeed/data/cameras.xml which seems to be the guidance (data/cameras.xml) for the file.

robcampbell95 avatar Jan 06 '23 18:01 robcampbell95

The Z 9 is supported in the just released darktable 4.2 OOTB, except the HE modes.

kmilos avatar Jan 06 '23 19:01 kmilos