New feature: DNG raw image support for arv-viewer + 16 bit Bayer grayscale preview

[victhor393]

trafficstars

This PR adds support for saving raw images as DNG, and also enables Bayer 16 bit format support on arv-viewer.

Since Gstreamer still doesn't support Bayer formats with more than 8 bits, this PR adds a hack that treats Bayer 16 bit formats as grayscale. This mainly makes it possible for arv-viewer to save 16 bit images, while the preview window can be used to eyeball exposure settings and adjust composition.

DNG support is very much what I'd consider a work in progress. Right now, it's able to write Bayer 8 bit and 16 bit formats (tested with both the fake camera and a real camera with Bayer16 support), with no metadata, appropriate color correction matrices, profiles or white balance ("As Shot Neutral").

It uses libtiff to generate a valid DNG file (as far as Rawtherapee and Adobe's own DNG converter are concerned), instead of the Adobe DNG SDK, since getting the latter to compile under Linux seems rather difficult.

This PR has a number of issues: no Exif metadata is written with the DNG file, thumbnail isn't written to the file by libtiff, there's no error handling for the DNG writer function, and overall I'm not very confident in the quality of its code.

That being said, it does show potential usefulness:

• DNG can be viewed by pretty much all raw editors available today, unlike the "unformatted" raw currently supported by arv-viewer.
• Since DNG supports Exif metadata, we can write a variety of information about shooting conditions, camera and lens (if applicable) used, possibly making use of the "MakerNote" tag to add more information not supported by the Exif standard, but often present in industrial cameras (gain settings, internal camera temperature...)
• Provides a use case for the "camera quirks" feature, which could be used to implement per-camera (or sensor) color correction matrices and color profiles; gain scale calibration according to ISO sensitivity standard; bias frame subtraction; lens controls and parameters; etc.

I'm sending this PR in hopes that I can gather some useful comments regarding this new feature and how it can be further developed, so that it can be merged into Aravis eventually if so desired.

[EmmanuelP]

Hi @victhor393

Thanks for your work on this. This would be a neat addition. Some comments / question:

• Why DNG and not TIFF ?
• The fallback for format not supported by gstreamer is interresting, I guess it could be extended to all the pixel formats, using the number of bit per pixel given by the pixel format 32 bit value.
• The dependency on libtiff should be made optional, enabled by default if found
• If libtiff support is enabled, it should be the default format

[victhor393]

Hi @EmmanuelP,

Thank you for your comments.

Regarding the choice of DNG over TIFF, it was made because TIFF does not really support images with Bayer format. There was an extension added to the TIFF standard called TIFF/EP that adds support for such images, but it was never widely adopted, according to a brief research.

DNG is an attempt to extend upon the TIFF/EP work which has been widely accepted. Since the specification is public, it has been implemented by various image editors and viewers, including open source software such as RawTherapee and Darktable, which makes it a good choice for Bayer-formatted images, especially since other raw formats are proprietary.

It's also flexible enough to support non-standard color filter arrays, like sparse CFAs and array geometries other than 2x2, if required in the future, though no existing software supports processing such unusual images, to my knowledge.

Additionally, DNG supports "linear" (demosaiced) images, so it can be used for grayscale and RGB images as well. However, in my view, it offers no advantage over a standard TIFF in this situation, so it would be good to implement TIFF support for non-Bayer images.

While one could always convert the unformatted raw into a DNG after capture (which is what I'm used to doing), this approach offers a significant increase in convenience over the alternative of running a separate conversion program, taking notes of camera parameters and writing these to the DNG file using exiftool.

I will try to implement these other features you suggested and update the PR as they're completed.

[victhor393]

Is it acceptable to use arvfeatures.h for viewer-related features?

[EmmanuelP]

Is it acceptable to use arvfeatures.h for viewer-related features?

I would prefer a new arvviewerfeatures.h for now. May be at some point we could offer an helper in aravis library that will convert an ArvBuffer to a TIFF/DNG buffer, but let's keep it private until things stabilize.

[victhor393]

I updated my PR with an implementation of arvviewerfeatures.h.

[EmmanuelP]

Hi @victhor393

Sorry for the late review. I have tried your patch. When I'm trying to open the file in darktable, I'm only getting a black image. It is probably related to these GStreamer warnings:

Sources/aravis/build on  victhor393-feature-dng [$] via 🐍 v3.11.4 took 8s ❯ ./viewer/arv-viewer-0.8

(arv-viewer-0.8:495453): GStreamer-WARNING **: 06:53:52.194: 0.10-style raw video caps are being created. Should be video/x-raw,format=(string).. now.

(arv-viewer-0.8:495453): GStreamer-Video-CRITICAL **: 06:53:52.198: gst_video_scaler_2d: assertion 'src != NULL' failed

(arv-viewer-0.8:495453): GStreamer-CRITICAL **: 06:53:52.198: gst_sample_get_buffer: assertion 'GST_IS_SAMPLE (sample)' failed

(arv-viewer-0.8:495453): GStreamer-CRITICAL **: 06:53:52.198: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(arv-viewer-0.8:495453): GStreamer-CRITICAL **: 06:53:52.198: gst_mini_object_unref: assertion 'mini_object != NULL' failed

The compilation also emits warnings:

../viewer/arvviewer.c: Dans la fonction « _save_arv_buffer_to_dng_file »:
../viewer/arvviewer.c:973:57: attention: le passage de l'argument 2 de « TIFFWriteScanline » transforme un entier en pointeur sans transtypage [-Wint-conversion]
  973 |                         TIFFWriteScanline (tif, map.data[row * (width >> 2)], row, 0);
      |                                                 ~~~~~~~~^~~~~~~~~~~~~~~~~~~~
      |                                                         |
      |                                                         guint8 {alias unsigned char}
Dans le fichier inclus depuis ../viewer/arvviewer.c:36:
/usr/include/tiffio.h:437:47: note: « void * » attendu mais l'argument est de type « guint8 » {alias « unsigned char »}
  437 | extern int TIFFWriteScanline(TIFF* tif, void* buf, uint32_t row, uint16_t sample);
      |                                         ~~~~~~^~~
../viewer/arvviewer.c:998:41: attention: passer l'argument 2 de « TIFFWriteScanline » abandonne le qualificatif « const » du type pointé [-Wdiscarded-qualifiers]
  998 |                 TIFFWriteScanline (tif, &data[row * width * (bpp >> 3)], row, 0);
      |                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/tiffio.h:437:47: note: « void * » attendu mais l'argument est de type « const char * »
  437 | extern int TIFFWriteScanline(TIFF* tif, void* buf, uint32_t row, uint16_t sample);
      |                                         ~~~~~~^~~
../viewer/arvviewer.c:844:21: attention: variable inutilisée « caps_string » [-Wunused-variable]
  844 |         const char *caps_string;
      |                     ^~~~~~~~~~~
../viewer/arvviewer.c:842:24: attention: variable inutilisée « model » [-Wunused-variable]
  842 |         char *vendor, *model;
      |                        ^~~~~
../viewer/arvviewer.c:842:15: attention: variable inutilisée « vendor » [-Wunused-variable]
  842 |         char *vendor, *model;
      |               ^~~~~~
[11/11] Generating src/Aravis-0.8.typelib with a custom command

[victhor393]

Thanks for testing it, I haven't been able to keep up with this patch, as I had a lot of work to do for the last month or so. I'll try to work on it as soon as I have some time available.

[victhor393]

Sorry, it took much longer than I hoped for me to have the time to work on this again!