Inconsistencies between Larch and Athena/Artemis

[killiantallman]

I am seeing significant differences between the EXAFS spectra data from Larch and Athena as well as differences in the fitting results. I assumed that there would be minor differences between fitting with these two programs but I believe what I am seeing is too substantial.

I read the data in through the read_athena command which works perfectly. In Larch GUI I can see all of the data attributes and the parameters that came with the data file from Athena. I initially thought it was an issue with a parameter not being read in properly but they are all consistent (edge energy, normalization parameters, etc.).

Attached you will see 3 different chik plots at the three different kweights which are comparing the plotted data from Larch and Athena. Some minor spectral differences are observed, especially at low k. Now, as I am sure you know, these small discrepancies can have significant differences in the EXAFS spectra. To show you the big difference in r, I attached another image showing unwanted peak intensity and peak splitting variations.

I thought it might have something to do with the nfft or kstep settings in the xftf command where Larch is using too many or not enough data points relative to Athena. I tried playing around with the values for the transform but no ground was gained.

Any help on this issue would be greatly appreciated - Thank you in advance!!

kweight = 1 kweight = 2 kweight = 3 kweight = 2

[newville]

@killiantallman There are some differences in the details of how autobk works in Larch and Ifeffit . Some of these differences are definitely improvements. ;). For sure, the value of "clamp" is different. And (it turns out), the implementation of the basis spline is actually different in numerical detail. It should be sufficient in practice but it may be slightly different at the margins. Still, the differences you see look rather larger than they should be.

As it turns out, there was another recent question on roughly the same topic and in looking at that case I noticed one or two things that I think can be improved. First, the conversion of rbkg to "number of knots in the spline" (the historical +1 or +2 debate) can give different results. I also think that Ifeffit's autobk may have fitted slightly past "rbkg" (at the rounding level). This is a bit hard to notice in Ifeffit/Athena, but it becomes more obvious in Larch/Python as it also estimates the uncertainty in chi(k) based on the uncertainties in the background spline knots.

Anyway, I've been playing around with this on and off for the past week or so. I think the situation is improving -- just to be clear, I'm trying to get this "right" not reproduce Ifeffit's flaws.

Can you send a small project file with these data sets? Thanks,

[killiantallman]

@newville Please see the single data set attached for example, which is plotted in all of the figures above. Thank you for looking into this!

measurement1_1.zip

[cwgaldino]

I have recently encountered the same problem. Background removal from larch and Athena lead to significant different results. In addition, I noticed that a small change in rbkg leads to a significant change in the first coordination shell in larch, but not in Athena. See image below.

Files for a minimal working example are attached: sample_1.zip

[newville]

@killiantallman @cwgaldino I think these problems with autobk() are now much better in the github master branch. The results are not identical to athena/ifeffit, but they are closer and better than they were. I also now understand some of the remaining small differences and am sticking with the current larch/python version as better for actually having low chi-square values and from the ability to propagate the uncertainties to a reasonable estimate of delta_chi(k) that can then be used as the uncertainty in chi(k) in fits.

The sample script from @cwgaldino now gives the plot below:

which is definitely better for this example, and seems to work fine for me for other example data sets as well.

So, thanks for bringing this up and providing examples!

I have a couple more small changes that I want to make to XAS_Viewer and XRF fitting before pushing out a new release, but I'm aiming for later this week or early next week.

[newville]

@killiantallman @cwgaldino When you get a chance, please try out 0.9.47 and let me know if you think this is improved enough or if you are still seeing significant differences.

[killiantallman]

@newville I am not sure what you are seeing on your end but unfortunately I am still seeing significant differences. See the figure below of the same data with a undesired split in the first shell peak. The split is different from previous Larch versions, interestingly, but still does not reflect the data as Athena does.

[newville]

If I read in the Ni spectrum from measurement_1.prj and use default parameters for normalization and default parameters for background subtraction except that I set kweight=2, say with:

project = read_athena('measurement1_1.prj')
data = extract_athenagroup(project.measurement1_1)

pre_edge(data, nvict=0, nnorm=2)
autobk(data, rbkg=1.0, e0=8351.20, kweight=2, clamp_lo=0, clamp_hi=1)
xftf(data, kmin=2.5, kweight=2, dk=4, window='Kaiser-Bessel')

plot_chik(data, show_window=False, kweight=2, win=1, title="measurement1_1")
plot_chir(data, show_mag=True,  win=2, title="measurement1_1")

I see these:

[killiantallman]

@newville Okay great! I am now seeing what you are seeing. I assumed that the read_athena and extract_athenagroup commands are able to read in all of the parameters as they are from Athena. This doesn't seem the case though because I cannot reproduce your last plot (desirable data plot) without having this as a bare minimum script:

project = read_athena('measurement1_1.prj')
data = extract_athenagroup(project.measurement1_1)

pre_edge(data)
autobk(data, kweight=2)
xftf(data, kmin=2.5, kweight=2, dk=1, window='Kaiser-Bessel')

plot_chik(data, show_window=False, kweight=2, win=1, title="measurement1_1")
plot_chir(data, show_mag=True,  win=2, title="measurement1_1")

which is interesting because it means that the pre_edge(data) command and the specification of the kweight in the autobk command are essential to seeing correct data. All the other inputs seemed to not effect the spectra but removing those two aforementioned ones did. Athena does save a default kweight in the software but I guess that parameter is not read into Larch because I do not see it in the list of parameters using the show(data) command. Nevertheless, I can't guess why the pre_edge(data) command is required because I believe the Athena read in commands bring pre edge parameters and such. Very interesting.

I now have a working script though to analyze data.

So sincerely thank you very much for all of the help!!!

[cwgaldino]

@newville I have tried out version 0.9.47. Indeed, there was some improvement. I would say this is improved enough for this particular data set.

Just for fun, I tried using higher values of rbkg. This is the result:

For higher values of rbkg, we still see some significant difference between Larch and Athena. At first, such high values of rbkg are unrealistic for this data set. However, it would still be nice to understand why this happens.

I also tested smaller values of kweight in xftf(). Particularly, for kweight=1 we got this:

The general shape of the curves are fine, but curves from Athena and Larch diverge for higher rbkg.

I do feel like this is not a problem, since the curves for "realistic" rbkg's are fine. However, I think these differences between Athena and Larch worth discussing.

Files for a minimal working example are attached: sample_1.zip

Unrelated comment: It is funny how Athena tends to pair up curves with similar rbkg (probably due to some kind of approximation). The figure below is a close-up of top part of the first coordination shell of previous figure: