PollyNET
Clean up main programm
The main program includes now more than 5000 lines which are very good commented and easy to read.
However, there are a lot of repetitions which should be sourced out to sub routines to improve readability and the implementation of new products.
Thus, I will start to "clean up" picassoProcV3.m.
@ZPYin do you consider this as a new project, i.e. should be labelled as Picasso 4.0?
Yes, 5000 lines crowded in a file looks a little bit horrible. I also noticed this issue after I extended main function to a very large scale and can only use "Find..." option to navigate the main code instead of scrolling up and down.
Refractoring the code and breaking the code fragments with multiple repetitions into sub-routines/sub-functions would mitigate this problem. Below is one option to achieve this goal. Let's take the refractoring of particle depolarization ratio calculation for example.
The 200 lines of code were used for retrieving particle depolarization ratio from volume depolarization ratio. This calculation relies on the output from Klett/Raman retrievals. Therefore, you can save the output to data struct before the section of particle depolarization ratio retrieval, and use the data struct as input argument for pollyPDRModule.m (Let's assume this is name of sub-routine/sub-function for capsuling the code). You can just copy/paste the code in the main function to the sub-function and replace it with the sub-function calling. In addition for the sub-function, you should also add input arguments of PollyConfig and PollyDefaults.
Similar things can be done for Klett/Raman and quasi- retrievals as well. But I guess you would still have 1-2 k lines of code in the end... Then you would decide whether this is acceptable or try to refractor the code further.
In terms of starting a new project (Picasso v4.0), I cannot decide it. My suggestion is to put this fixes under Picasso 3.x. Because this is still compatible with previous versions and following the same logic follow as before, even if new products were introduced. We don't really need to re-structure anything. Frequent major increment is not a good thing, at least it gives others an impression of instability.
My concern is issue #194. If using the outputs from target classification into data averaging, this would create a loop for the logic flow. I know this idea has been applied in other processing chain (see [1]), but this is different than the current idea. And this would even bring new features and possibilities for the data processing. I would suggest to name this as a major version increment and re-design the main function if possible.
Referemces
[1] Thorsen, T. J., Fu, Q., Newsom, R. K., Turner, D. D., and Comstock, J. M.: Automated retrieval of cloud and aerosol properties from the ARM Raman lidar. Part I: Feature detection, Journal of Atmospheric and Oceanic Technology, 32, 1977-1998, 10.1175/JTECH-D-14-00150.1, 2015.
