Include cloud-screened optical propertiy profiles

[HolgerPollyNet]

For certain locations, like at Cabo Verde, it might be useful to screen out boundary layer clouds but then average the remaining lidar slices to obtain the vertical profiles of optical properties.

Thus, we want to implement this option.

Proposed steps:

1. Use the high-resolution cloud mask from the target categorization.
2. Flag ALL profiles (time wise) as invalid which contain clouds
3. (Maybe a height threshold has to be included)
4. Apply the usual Picasso on the resulting data using all remaining profiles --> force Picasso to use several profiles interrupted by cloudy profiles. I.e. relaxing the minIntprofile config attribute to larger time frames.
5. testing
6. implementing this as OPTION (i.e. by using a flag in the config) so that it is not done for all stations.

[ZPYin]

There is another possibility for doing this, instead of using the cloud mask from target classifications. I once implemented a cloud detection function (cloudDetect_Zhao.m for high resolution cloud masks), including planetary boundary layer clouds mask. This could be used to avoid introducing the loop of the data processing.

You can activate this cloud mask functionality by turning cloudScreenMode to 2.

For more information in terms of the cloud mask algorithm, you can refer to [1]. I also used this method for long-term polarization lidar data to screen out clouds. It works quite well.

References

[1] Zhao, C., Wang, Y., Wang, Q., Li, Z., Wang, Z., and Liu, D.: A new cloud and aerosol layer detection method based on micropulse lidar measurements, Journal of Geophysical Research: Atmospheres, 119, 6788-6802, 10.1002/2014JD021760, 2014.

[elmarinou]

I have some python functions with filters for cloud screening based on L2 extiction, particle depolarization, and cloud visinity criteria (wich was build originally for DLR halo data, and tested briefly for polly data in Earlinet retrievals). I can contribute with providing the functions, making a presentation also. but I am afraid I will not find the time to implement it to the pollyphon algorithm, any time soon.

[HolgerPollyNet]

Great! Maybe this could then be a specific webinar topic and later on we try to implement? @elmarinou

[elmarinou]

yes, we could do this :)

[HolgerPollyNet]

Hey @HenniHG,

are you still working on this topic? @georgmueller is also starting to work on the cloud screening. Maybe you can discuss together?

The first step to work on would be to understand the current cloud screening at this location: https://github.com/PollyNET/Pollynet_Processing_Chain/blob/e85bef047e7cd642a52ab25d1288fd07bd924076/lib/interface/picassoProcV3.m#L709

Then think about improvements or options how one could average between.