NCAR
Support for user-defined assimilation order among observation types
Hello DART team,
I would like to ask whether DART currently supports specifying an explicit assimilation order among different observation types within a single run of filter.
For example, I would like to perform assimilation in the following sequence (in the same assimilation window):
First assimilate all TEMPERATURE observations
Then assimilate SATELLITE_BLENDED_SST observations
My understanding from the documentation and experiments is that DART’s EnKF/EAKF framework performs a simultaneous update of all observations, and therefore does not allow users to control the assimilation order within one filter call.
Currently the only workaround seems to be:
Run filter once with only TEMPERATURE observations
Use the resulting analysis as the prior
Run filter a second time with SATELLITE_BLENDED_SST observations
This two-step approach works, but it is computationally expensive and complicates workflow automation. Thank you!
Greetings yxyyyyyy thanks for your interest in DART!
DART intentionally does not support the specification of the order of assimilation because the "square root" algorithms (EAKF, EnKF, ...) are fundamentally linear, so the increments from the various observations can be added together in any order. If you have some evidence that the order matters in this context, we'd be interested to see it.
I'm guessing that your work is in the research stage, and not operational, so proceeding to investigate with the 2-assimilation method is the most efficient way forward.
An additional complication that you'll need to handle is the inflation,
which is designed to be applied once at each assimilation time. If you apply it to each assimilation
your ensemble spread may be inflated the wrong amount (probably too much).
This is especially true if you have very different numbers of T and SATELLITE obs.
so you may need to turn it off for the assimilations which have significantly fewer obs.
I can't say what 'significantly' means here; that depends on the research goals
and other features of the assimilation.
Neglecting to do this correctly would result in the 2-assimilation method yielding different results from the combined single assimilation, but not because of the separation of observation types.
We don't "support" this but dart steps through the observation sequence in time order in a given assimilation window. So you could change the time of the observations so the TEMPERATURE observations were time t, and the SATELLITE_BLENDED_SST were time t+1s. So you force the order by setting the time.
We don't "support" this but dart steps through the observation sequence in time order in a given assimilation window. So you could change the time of the observations so the TEMPERATURE observations were time t, and the SATELLITE_BLENDED_SST were time t+1s. So you force the order by setting the time.
Thank you very much for your helpful reply. Your explanation was very useful.
I have one additional question: How should the observation timestamps be modified in DART in practice? Should the different observation types be assigned different time values when generating the obs_seq.out file? For example, by adjusting the days and seconds fields in the DART time structure?
If possible, could you briefly describe the recommended procedure or any key points to pay attention to? For instance:
At what stage should the timestamps be set? (During the script that generates the observation sequence, or through a specific DART utility?)
Do the observation times need to be strictly increasing, or simply different from each other?
Thank you again for your patience and guidance. I really appreciate your help, and I look forward to any further advice you may have.
Best regards,
We don't "support" this but dart steps through the observation sequence in time order in a given assimilation window. So you could change the time of the observations so the TEMPERATURE observations were time t, and the SATELLITE_BLENDED_SST were time t+1s. So you force the order by setting the time.
Thank you again for your helpful explanations in your previous message. I would like to confirm whether my understanding of the observation ordering and inflation behavior in DART is correct.
My current understanding is as follows:
Assimilation order and observation times DART assimilates observations according to the time stamps in the obs_seq.out file. For example, if two TEMPERATURE observations both have the time stamp 3600 144270, then they will be assimilated sequentially in the same time group, following the order in which they appear in the obs_seq.out file.
If I assign the SATELLITE_BLENDED_SST observations a later time stamp, such as 3601 144270, then DART will automatically place these observations in a later assimilation step, after all observations at 3600 144270 have been processed. Could you please confirm whether this interpretation is correct?
Inflation applied once per assimilation time My understanding is that DART applies inflation once for each unique assimilation time. Therefore, if SATELLITE_BLENDED_SST observations are assigned a different time (e.g., 3601 144270 instead of 3600 144270), then DART would treat this as a second assimilation time and would apply inflation again at this new time step.
Is this the correct behavior?
Thank you very much for your time and guidance. Your explanations have been extremely helpful for my research, and I appreciate any clarification you can provide on these points.
Assimilation order and observation times The DART tools create a linked list of observations in the obs_seq.out file, which is based on the time ordering of the observations, not on the physical order in the file. It's misleading to think of SAT obs at a later time as being in a different or separate assimilation step. DART assimilates them after the T obs, because the linked list is designed to do that, but all of the observations requested from an obs_seq.out file are assimilated in the same assimilation step. If a group of observations have the same time, then they will be ordered according to how the DART tools encountered them in the original observation files, but this sub-ordering doesn't matter.
Inflation applied once per assimilation time It's best to think of it as being applied once per obs_seq.out file. That file can have a range of times in it, but they are all assimilated in one step and inflation is applied once to this one step. If you put the 2 observation types in separate obs_seq.out file, and make filter process them both, then inflation will be applied twice (unless you prevent it using fancy scripting).