NCAR
Inflation not turned off while using obs_impact tool
Describe the bug
- List the steps someone needs to take to reproduce the bug.
CLM-DART is used to assimilate observations of leaf area (monthly frequency), SIF (monthly frequency) and SWE (daily frequency). We use the obs impact tool to isolate the impact of the leaf area (LAI) observations onto leaf carbon only, the impact of SIF observation onto leaf nitrogen only, and the impact of SWE observations on snow only. During the daily assimilation time step where there is only SWE observations the ensemble spread of the leaf carbon and nitrogen gets exceedingly large because inflation is still applied to these variables. The inflation damping value is set to 0.9 so the inflation >1 is applied for all days of the month. There are no observations to counteract this increased spread for leaf carbon and leaf nitrogen. This spurious spread causes poor forecast performance.
This bug was already mentioned in a separate and concurrent UPDATE and NO_COPY_BACK inflation issue #775
- What was the expected outcome?
Because we use the obs impact tool such that leaf carbon and leaf nitrogen are not influenced by SWE observations (and daily obs only include SWE observations) the expectation was that no inflation would be applied during these times to leaf carbon and leaf nitrogen (inflation =1). In other words, when variable localization is applied such that observation A does not influence state variable B, if observation A is the only observation available during a particular timestep, inflation should not be applied (inflation =1) for that state variable B.
- What actually happened?
What actually happens is that inflation is applied for all timesteps for all variables in the DART state regardless of the obs impact tool specifications. This causes huge ensemble spread for leaf nitrogen, and also for leaf area (shown below), because no observation constraint is available to counteract the effect. This produces spurious behavior including: 1) high leaf nitrogen generates high photosynthesis, and 2) large spread in leaf area also promotes the dieoff (LAI =0) of certain ensemble members which destroys the background error covariance.
Screenshots
In below figures the 'LAI SIF DA' simulation gives the expected ensemble spread, whereas 'LAI SIF SWE DA' simulations give the spuriously large ensemble spread. Inflation damping value set to 0.9.
Which model(s) are you working with?
CLM5
Version of DART
v11.8.5-14
Have you modified the DART code?
Quick fixes: Manually adjust the input.nml such that for assimilation time steps which only have SWE observations, the leaf carbon and leaf nitrogen state variables are set as 'NO_COPY_BACK' (or these variables are removed from the DART state) so that inflation is not applied. This requires custom/dynamic changes to the CLM shell scripting which must be aware of the time/date.
Bad fix: Perhaps we could adjust the inflation settings such that the damping value is much lower (0.1) thus dramatically decreasing the inflation, however, we would prefer the damping values for the leaf carbon and leaf nitrogen to be different than for SWE. Currently there is no way to apply state variable specific inflation settings so this solution is not ideal.
Permanent fixes: The DART code should recognize the obs_impact tool by automatically setting inflation =1 if there is no relevant observation at that time step on a per variable basis. I think this is the same behavior for spatial localization, where inflation is dampened/removed to reflect the localization distance. At the very least, the documentation has to improve on this -- there is no mention of how inflation interacts with the obs_impact_tool. It's very easy to miss this behavior, unless the user is looking at the ensemble member specific behavior.
Thanks for submitting this report which clearly outlines the concern.
In my opinion, this is not a bug nor is it an undocumented feature. The obs_impact tool is designed to limit the impact of observations in updating the prior distribution of a state variable and its inflation. This is done consistently. The function cov_and_impact_factor in assim_tools_mod computes the product of the localization and the obs_impact and it is named ‘final_factor’ when returned. This product is used consistently in both subroutines obs_update_ens and update_varying_state_space_inflation to limit the impact of the observation.
This mechanism is not supposed to limit the application of inflation to state variables during an assimilation step and it does not. This functionality would not be well-defined since there could be observations of multiple kinds of observations being assimilated and each could have a different setting for no_obs_impact.
If I understand correctly, the problem here is that this application has two distinct assimilation processes going on. One occurs at high frequency, maybe daily, while the other is at low frequency, maybe monthly. The desire is to not change the low frequency state variables with the high frequency observations. I think that the solution already in place for this, making these variables no_copy_back when they are not to be updated in any way, is the correct approach.
On Fri, Nov 22, 2024 at 3:21 PM Brett Raczka @.***> wrote:
Shout out to @XueliHuo https://github.com/XueliHuo for identifying this issue.
— Reply to this email directly, view it on GitHub https://github.com/NCAR/DART/issues/777#issuecomment-2494968458, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDHUIVNMMWHB2BZVSNH2DT2B6U5RAVCNFSM6AAAAABSKIVDCKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJUHE3DQNBVHA . You are receiving this because you are subscribed to this thread.Message ID: @.***>
Yes -- I think you're understanding is correct, although I don't see where in the documentation a description of how the obs_impact tool is intended to interact with inflation. For example, inflation is not mentioned at all within the obs_impact tool section. I may be revealing my ignorance of how the code is supposed to work, but I don't see how the existing code limits the impact of inflation on state variables that are completely localized from the observations -- it appears the inflation is just tapered down according to the inflation 'damping' factor. Is this the intended functionality? When a state variable experiences no increment from an observation because of either spatial or variable localization, it seems like inflation should be prescribed a value = 1, and should not have memory of the prior inflation state. Maybe applying a different inflation flavor (different than 5) for this application is preferred.
The key distinction here is between the application of inflation (inflating state variable priors) and the updating of the adaptive inflation (the impact of the observations on the inflation prior estimate). The second is controlled by obs_impact, the first has basically nothing to do with obs_impact. Perhaps the solution is to update the obs_impact documentation to be explicit about it only affecting the impact of an observation on the prior state ensemble and the prior inflation distribution.
On Mon, Nov 25, 2024 at 9:36 AM Brett Raczka @.***> wrote:
Yes -- I think you're understanding is correct, although I don't see where in the documentation a description of how the obs_impact tool is intended to interact with inflation. For example, inflation is not mentioned at all within the obs_impact tool section. I may be revealing my ignorance of how the code is supposed to work, but I don't see how the existing code limits the impact of inflation on state variables that are completely localized from the observations -- it appears the inflation is just tapered down according to the inflation 'damping' factor. Is this the intended functionality? When a state variable experiences no increment from an observation because of either spatial or variable localization, it seems like inflation should be prescribed a value = 1, and should not have memory of the prior inflation state. Maybe applying a different inflation flavor (different than 5) for this application is preferred.
— Reply to this email directly, view it on GitHub https://github.com/NCAR/DART/issues/777#issuecomment-2498503707, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDHUIUUV3XCGCE4AE335X32CNGXTAVCNFSM6AAAAABSKIVDCKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJYGUYDGNZQG4 . You are receiving this because you commented.Message ID: @.***>
Brett, if the state variable experiences no increment, the inflation is not updated either. When the state is not updated, the prior remains unchanged. The inflation algorithm is Bayesian and recursive in time, so if there is no update then we are left with the prior inflation distribution. Right now, the only way to reduce the impact of inflation in case of no update is through damping. Setting the inflation to 1 breaks the recursive nature of the scheme. I think you'll need to adapt the damping to your observation network frequency.
OK -- so my interpretation is the obs_impact tool is completely unrelated to the application of inflation, it simply turns off the adaptive nature of the inflation update (Bayesian, recursive nature) but still applies the inflation.
The use of damping in this case to address this issue is not ideal -- because it is my understanding the damping factor is applied universally across all DART state variables, so, in this case, that would not be appropriate, because of the different frequency of the observations. In the case where we want to turn off the recursive nature of the inflation update, toggling to 'NO_COPY_BACK' to immediately set the inflation = 1 seems preferred.
Another question -- in cases of spatial localization, where the state variable increments are downscaled based on the Gaspari-Cohn radius, how does inflation interact with the application of spatial localization? My impression is that the inflation would also be downscaled?
Brett, yes we apply prior inflation before we update its value (mean and sd)! Inflation is spatially localized just like the state. As you move away from the observation, the localization impact increases until you get to impact at all far away from the obs location.
The same reducing factor, which is a product of the Gaspari Cohn and the standard localization, is applied to the increments for both the prior state_variable ensemble and the prior inflation parameters for the state variable. See my earlier comment about the routine where this final_factor is computed.
On Mon, Nov 25, 2024 at 10:46 AM Brett Raczka @.***> wrote:
Another question -- in cases of spatial localization, where the state variable increments are downscaled based on the Gaspari-Cohn radius, how does inflation interact with the application of spatial localization? My impression is that the inflation would also be downscaled?
— Reply to this email directly, view it on GitHub https://github.com/NCAR/DART/issues/777#issuecomment-2498661873, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDHUITDQF4ZLUB7PU7CMW32CNO6HAVCNFSM6AAAAABSKIVDCKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJYGY3DCOBXGM . You are receiving this because you commented.Message ID: @.***>
OK -- my final question. Is the reducing factor (localization of inflation) applied the same way to spatial localization and variable localization? In other words, is any localization (reducing factor) applied to inflation when using variable localization -- which is controlled through the obs_impact tool ? .
Localization (final_factor in the code) is computed as the product of spatial localization and variable localization (obs_impact_tool) as Jeff mentioned. So, yes variable localization is accounted for when updating inflation.
For more details, look for cov_and_impact_factors in assim_tools_mod
Again, see my original reply above which I think addresses your question. The reducing factor that incorporates all modifications to the regression coefficient is computed once and passed to the routine that updates the prior state ensemble and also to the routine that updates the prior inflation parameters. You can follow this with a peek inside assim_tools_mod with a search for final_factor.
On Mon, Nov 25, 2024 at 11:07 AM Brett Raczka @.***> wrote:
OK -- my final question. Is the reducing factor (localization of inflation) applied the same way to spatial localization and variable localization? In other words, is any localization (reducing factor) applied to inflation when using variable localization -- which is controlled through the obs_impact tool ? .
— Reply to this email directly, view it on GitHub https://github.com/NCAR/DART/issues/777#issuecomment-2498707562, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDHUISPPVOQRS5JYL4IZX32CNRNHAVCNFSM6AAAAABSKIVDCKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJYG4YDONJWGI . You are receiving this because you commented.Message ID: @.***>
Thanks @jlaucar and @mgharamti. Even though variable localization is accounted for when updating inflation, it seems to me there is nothing in the code that enforces the condition that prior inflation = 1, when we know the increment = 0. This condition can only be enforced through assigning variables as 'NO_COPY_BACK'.
The original theoretical motivation for the adaptive inflation is predicated on the idea of an observing system that is homogeneous in time, but not necessarily in space. One can argue that this can be extended to cases where the observing system is slowly-varying in time. The inflation damping, which has never been adequately studied and documented to the best of my knowledge, was introduced to deal with the case where the observing system is inhomogeneous in time, in particular when observations can quickly disappear in time. Everything else being equal, the damping would be the implemented solution to the original problem here I think. However, if I understand correctly, the damping settings are not quick enough to avoid blowing up and some inflation may be needed when the infrequent obs are next available.
Automating a general solution to this could be challenging. For instance, how does one know that no observations are impacting a given type of state variable. Requiring a completely filled out obs_impact table that required all possible obs and their impact on all possible state variables is the only robust solution that I can think of, but I'm not sure we want to require that since the vast majority of applications don't use a table at all. It may be that your current solution is pretty much the best available.
On Mon, Nov 25, 2024 at 1:26 PM Brett Raczka @.***> wrote:
Thanks @jlaucar https://github.com/jlaucar and @mgharamti https://github.com/mgharamti. Even though variable localization is accounted for when updating inflation, it seems to me there is nothing in the code that enforces the condition that prior inflation = 1, when we know the increment = 0. This condition can only be enforced through assigning variables as 'NO_COPY_BACK'.
— Reply to this email directly, view it on GitHub https://github.com/NCAR/DART/issues/777#issuecomment-2498963821, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDHUIV6PPPNMXZGAO3F3Z32COBVZAVCNFSM6AAAAABSKIVDCKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJYHE3DGOBSGE . You are receiving this because you were mentioned.Message ID: @.***>
Converting the issue "Inflation not turned off while using obs_impact tool" to a discussion. I can convert back to an issue (or open a new issue) if this is/becomes something that needs to be changed in the code/docs.
