hrv_clouds analogue for FCI?

[guidocioni]

In SEVIRI the hrv_clouds composite is constructed (correct me if I'm wrong) using

R - HRV channel
G - HRV channel
B - IR10.8 channel 

(https://github.com/pytroll/satpy/blob/c245caba088358fe01d53983b5f3b9275d114baa/satpy/etc/composites/seviri.yaml#L415-L423)

I believe Eumetsat has decided not to create an analogue for FCI, as the currently advertised composites are mainly true_color and geo_color. This is fine for most use cases if one is not using the HRFI data at 0.5/1 km resolution. In this case we only have 4 channels so that these composites (I believe) cannot be computed. Currently i only see these composites available for FCI HR data

['colorized_ir_clouds',
 'geo_color_background_with_low_clouds',
 'geo_color_high_clouds',
 'geo_color_low_clouds',
 'geo_color_night',
 'ir108_3d',
 'ir_cloud_day',
 'ir_sandwich',
 'ir_sandwich_with_night_colorized_ir_clouds',
 'night_ir105']

However, an analogue composite to hrv_clouds could be computed using VIS0.6 and IR10.5 channels (of course with resampling as IR has only 1 km resolution). I've seen some websites that implemented this already, e.g. https://meteologix.com/fr/satellite/italy/satellite-hd-10m-superhd/20241017-0840z.html

Is this something that could be implemented in satpy?

[pnuu]

You can use HRFI and FDHSI data together. Satpy will pick and use the channels with the highest resolution that are available.

See my back-of-the-enveolope version HRV Fog here: https://github.com/pytroll/satpy/issues/2670#issuecomment-1846983104

Similarly you could create the HRV Clouds. Currently I'm not going to put these in operations at FMI, so haven't looked at a more streamlined approach which might need creating a new averaging compositor or such. Using the plain 0.6 um channel will not give equivalent HRV composites because it doesn't have the longer wave data that HRV channel has.

[gerritholl]

See also https://github.com/pytroll/satpy/issues/2670 where the table notes that HRV Clouds and HRV Fog are currently missing for FCI.

If you develop a composite that you think is of interest to a wider community, you are very welcome to propose it to Satpy as a PR.

[guidocioni]

Well actually I wanted to avoid using HRFI and FDHSI together (because it would mean downloading double the amount of data). My idea was to get a composite as close as possible to hrv_clouds using only high resolution data (0.5 km for the visible part and 1 km for the infrared part).

What is the easiest way to obtain something like that?

I'm not sure what was used to obtain the image of the first post but it seems to be higher resolution than the hrv_clouds of SEVIRI, albeit having similar colors.

[guidocioni]

Apparently, just using the same recipe from SEVIRI works pretty well...

(fci.yaml)

hrv_clouds:
  compositor: !!python/name:satpy.composites.GenericCompositor
  standard_name: hrv_clouds
  prerequisites:
    - name: vis_06
      modifiers: [sunz_corrected]
    - name: vis_06
      modifiers: [sunz_corrected]
    - ir_105

Here is a comparison of SEVIRI and FCI.

Notice I'm only using HRFI data for this, no need for FDHSI. I'm using the native resampler and then cropping to the plotting area, which means that (in theory) everything should be resampled to the highest resolution (0.5 km).

I'm not sure I get why this should not be correct, besides the fact that obviously these channels do not have the same wavelength as in SEVIRI. Wouldn't be bad to include this in the main code.

[gerritholl]

It's not wrong, just different. At least at DWD, we have not decided yet whether we will do an attempt to simulate SEVIRI looks with FCI channels, or train our users and tell them some RGBs look different now. Either way, the calibration of FCI is not yet finalised, so it's too early for final RGB finetuning.

As for the name, I wouldn't call the FCI composite hrv_clouds, but rather vis06_clouds or similar. It doesn't contain HRV so it shouldn't be called HRV, I think.

[ameraner]

Well, taking "HRV" by the letter as "High Resolution Visible", I guess the FCI VIS0.6 at 500m qualifies as that channel, since it's again the highest resolution visible channel on the instrument, as in SEVIRI:) Of course, the FCI VIS0.6 is very different compared to SEVIRI HRV, but at the same time hrv_clouds is an established name and could be kept for the sake of simplicity/continuity.

So I would actually support adding Guido's recipe to satpy, lots of users have been asking about it. Whether it's still a good idea to use this composite or others would be better is a different discussion, but I think Satpy could still offer it.

cc @strandgren

[guidocioni]

Sorry i really didn't mean to sparkle a naming debate 😂.In my case it was really convenient to use the same name because it means I can transparently plot the same composite in my scripts with either SEVIRI or FCI data and obtain two virtually equivalent images. Or let's say it was just laziness...

[ameraner]

Naming debates are our favourite hobby around here :D I agree to keep things simple and keep the same name, as it will lead to much easier acceptance and straightforwardness. We can also mention this at the next RGB Workshop, and if a new name is agreed on, we can have that as well in parallel.

[guidocioni]

Sorry @ameraner could I disturb you with two small questions about this?

• How does the compositor knows that it has to invert the ir channel? In the YAML there is nothing about this but it still does it
• Is there any way to improve the contrast so that bright areas are not as bright (hope you know what I mean) :) or maybe is it because I'm plotting such a small area that the normalization is done on a limited interval?

Thanks!

[gerritholl]

@guidocioni The inversion is done by the enhancer and defined in the enhancement:

  hrv_clouds:
    operations:
      - name: stretch
        method: !!python/name:satpy.enhancements.stretch
        kwargs:
          stretch: crude
          min_stretch: [0, 0, 323]
          max_stretch: [100, 100, 203]
    standard_name: hrv_clouds

Note that the stretch for the IR channel is min 323, max 203, which means it's inverted. See https://github.com/pytroll/satpy/blob/main/satpy/etc/enhancements/generic.yaml#L1071-L1079.

To improve the contrast, one could tweak those numbers.

[guidocioni]

@guidocioni The inversion is done by the enhancer and defined in the enhancement:

  hrv_clouds:
    operations:
      - name: stretch
        method: !!python/name:satpy.enhancements.stretch
        kwargs:
          stretch: crude
          min_stretch: [0, 0, 323]
          max_stretch: [100, 100, 203]
    standard_name: hrv_clouds

Note that the stretch for the IR channel is min 323, max 203, which means it's inverted. See https://github.com/pytroll/satpy/blob/main/satpy/etc/enhancements/generic.yaml#L1071-L1079.

To improve the contrast, one could tweak those numbers.

Ah ok I see, this is done in the generic enhancements. I was looking for this entry in the seviri enhancements... So if I copy paste this block in etc/enhancements/fci.yaml I could tweak those numbers to change the contrast and this will overwrite the generic one?

[gerritholl]

So if I copy paste this block in etc/enhancements/fci.yaml I could tweak those numbers to change the contrast and this will overwrite the generic one?

Yes, satpy should find yours first and use that instead of the one defined directly in satpy. I'm not sure about the search order between what's in seviri.yaml or generic.yaml locally or in satpy, but if you put it in your own generic.yaml it should be found.

[pnuu]

The order in which the configs are taken to use is:

• the built-in "generic" ones ** visir.yaml for composites ** generic.yaml for enhancements
• sensor specific built-ins ** seviri.yaml for example
• the user defined configs

These are stacked and the one that remains is used.

[guidocioni]

Thanks guys, I stretched the upper limit of the visible to 140 (instead than 100) and it does look already much better

Normal range

Strecthed range

[ameraner]

Thanks for this @guidocioni! It does look much better - Did you maybe check how it compares to SEVIRI with the original recipe for this case? Do you see systematic differences due to the channel differences between the instruments?

I am all in for proposing improvements, but we should be careful when changing the recipes drastically, as some atmospheric situations might then look different, which will cause confusion for people used to the SEVIRI default (e.g. as described in the quick guides https://user.eumetsat.int/resources/user-guides/hrv-clouds-rgb-quick-guide). This could be another good use case for discussion at the upcoming RGB workshop in spring :)

[guidocioni]

I think we need some detailed investigation for many case studies done by people smarter than me, to get a meaningful conclusion :D

[ameraner]

If you agree, I would propose to push the FCI composite to satpy with the original recipe for now, as from your first example it replicates quite well SEVIRI, so the usual interpretation and guides still apply. Then we can work in parallel on an improvement, including the community in the loop - you can of course keep your own version in your own configs and use it! Or if you like even add it to satpy with another name, like hrv_clouds_dark or so. I will try to bring up the topic at our next meeting to put it in the agenda for further discussions.

[guidocioni]

Hey all, I'm still missing the hrv_clouds recipe in fci.yaml. Are you planning to push that sometime in the future? thanks

[mraspaud]

@guidocioni I don't think we are going to implement this ourselves: there is no need for it from our side, and on top of that, at the rgb workshop that just finished today, we decided to categorise it as a legacy rgb (better replaced with either true color or cloud phase I think).

However, if you have a need for it, PR is welcome.

[guidocioni]

@guidocioni I don't think we are going to implement this ourselves: there is no need for it from our side, and on top of that, at the rgb workshop that just finished today, we decided to categorise it as a legacy rgb (better replaced with either true color or cloud phase I think).

However, if you have a need for it, PR is welcome.

Sure, I've drafted a PR here https://github.com/pytroll/satpy/pull/3100. I don't know enough the project to understand whether this has any larger impact, or whether it needs additional tests (I don't think so). in any case I'm happy to contribute further if you need anything.

[gerritholl]

Can this be closed with the merger of https://github.com/pytroll/satpy/pull/3100 ?

[guidocioni]

Can this be closed with the merger of #3100 ?

indeed