flavio
flavio copied to clipboard
flav-io
Slightly larger SM uncertainties for RD(*) compared to literature
For a long time we have been using version 1.5.0, which produces a SM uncertainty for the observables Rtaul(B->Dlnu) and Rtaul(B->D*lnu) of 0.006 and 0.003 respectively, which are similar to the values published by HFLAV (https://hflav-eos.web.cern.ch/hflav-eos/semi/spring19/html/RDsDsstar/RDRDs.html).
However, we recently updated Flavio to version 2.2.0, and we have found that the SM uncertainties calculated by this new version are much larger, 0.087 and 0.031 respectively. If these values were true, the tension in the RD(*) observables would disappear (see attached figure).
We have examined the source code, and the functions used to compute RD(*) or the SM uncertainty have not changed since version 1.5.0. The only relevant change, as far as we know, could be the new B->D and B->D* form factors introduced in version 2.0.
Is this an intended behaviour of the code? Is there any way to produce uncertinty predictions similar to those of HFLAV in the newer versions of Flavio?
@Jorge-Alda thank you for reporting this. From flavio.sm_error_budget('Rtaul(B->Dlnu)') and flavio.sm_error_budget('Rtaul(B->D*lnu)') one indeed finds that there are the entries
('CLN lp_3(1)',
'CLN c_xi',
'CLN lp_6(1)',
'chi_2p(1)',
'eta(1)',
'chi_2pp(1)',
'CLN l_4(1)',
'CLN l_6(1)',
'etap(1)',
'CLN l_2(1)',
'CLN lp_4(1)',
'CLN xi3',
'etapp(1)',
'chi_3p(1)',
'CLN l_3(1)',
'CLN lp_5(1)',
'CLN lp_2(1)',
'CLN rho2_xi',
'chi_2(1)',
'chi_3pp(1)',
'CLN l_1(1)',
'CLN lp_1(1)',
'CLN l_5(1)'): 0.32750804819346513,
and
('CLN lp_3(1)',
'CLN lp_6(1)',
'CLN c_xi',
'chi_2p(1)',
'eta(1)',
'chi_2pp(1)',
'CLN l_4(1)',
'CLN l_6(1)',
'etap(1)',
'CLN l_2(1)',
'CLN lp_4(1)',
'CLN xi3',
'etapp(1)',
'chi_3p(1)',
'CLN l_3(1)',
'CLN lp_5(1)',
'CLN lp_2(1)',
'CLN rho2_xi',
'chi_2(1)',
'chi_3pp(1)',
'CLN l_1(1)',
'CLN lp_1(1)',
'CLN l_5(1)'): 0.13231184631260465,
that seem to be unreasonably large. These parameters are related to the new form factors implemented in https://github.com/flav-io/flavio/pull/92 (and one small bug in the initial implementation was fixed in https://github.com/flav-io/flavio/issues/96). At the moment, I am not sure what might cause this problem. @DavidMStraub do you have an idea?
This might also be because what is usually called "SM prediction" contains data from B->D(*)(e, mu)nu at low q². In the Bayesian language of 1908.09398, the problem might be the difference between the prior and posterior predictions for RD and RD*. Or there is simply a bug.
@dvandyk and @MartinJung1978 can you please comment whether those quoted large uncertainties for RD and RD* could be reasonable "prior" predictions (i.e. without using e & mu data) or whether you suspect there is a bug. In the latter case, it would be great if you could have a look at the code, taking into account the issue that was fixed in #96.
Our values in 1912.09335 without experimental input are R(D) = 0.2989±0.0032 and R(D*) = 0.2472±0.0050, so nothing like the numbers above. Without the inclusion of Bs, the uncertainties were not that different.
Thanks! So it's a bug. Sorry but I won't be able to help much in hunting it down this week.
I have adapted the code of sm_error_budget to modify all parameters except one, and compute the standard deviation of the predictions for Rtaul(B->D*lnu). The best results are obtained when CLN rho2_xi or CLN c_xi are left unmodified:
CLN rho2_xi 0.01381
CLN c_xi 0.01886
CLN xi3 0.02775
chi_2(1) 0.02599
chi_2p(1) 0.02601
chi_2pp(1) 0.02585
chi_3p(1) 0.02597
chi_3pp(1) 0.02598
eta(1) 0.02630
etap(1) 0.02630
etapp(1) 0.02587
CLN l_1(1) 0.02599
CLN lp_1(1) 0.02599
CLN l_2(1) 0.02601
CLN lp_2(1) 0.02052
CLN l_3(1) 0.02721
CLN lp_3(1) 0.02889
CLN l_4(1) 0.02599
CLN lp_4(1) 0.02599
CLN l_5(1) 0.02686
CLN lp_5(1) 0.02569
CLN l_6(1) 0.02584
CLN lp_6(1) 0.02620
If CLN rho2_xi and CLN c_xi are left unmodified at the same time, the standard deviation goes down to 0.00863.
As a temporal workaround, I created a fork to use the old form factors with the new code, and I think it works. If you manage to fix the bug, please let me know. Thank you!
I don't have time to look into this in detail, but my guess is that the bug resolved in #96 was fixed in 4ba46c1df8ae230c4a37434fd61ea5c61ba712cb only for the central values, while it has been forgotten (using passive voice to hide my responsibility) to fix the correlation matrix as well. @Jorge-Alda, it would be great if you could check this using the notebook linked in #96.
If I understand #96 correctly, the problem was that there was a wrong prefactor in c_xi = 2xi''. If that was the case, the correlation matrix should still be correct, since it is invariant when re-scaling the data.
Anyways, I'll check the notebook tomorrow.
The first row and first column of the correlation matrix have the wrong sign. With the correct matrix, the uncertainties for Rtaul(B->Dlnu) and Rtaul(B->D*lnu) are 0.008, which are more reasonable (although still a bit large)
Thank you @Jorge-Alda for the fix!
While now the order of magnitude is fine, the uncertainties are still twice as large as those quoted by @MartinJung1978. Since there was some discussion in https://github.com/flav-io/flavio/issues/96 about non-Gaussian effects, I have used the samples from https://zenodo.org/record/3376898 to directly compute RD and RD* and their uncertainties using the flavio implementation. Keeping all other input parameters at their default central values (i.e. having uncertainties only due to the form factors), I find (see this notebook)
R(D) = 0.2995 ± 0.0054
R(D*) = 0.2456 ± 0.0079
If instead of using the mean and the standard deviation, I use a Gaussian KDE to get the central values and errors of the distributions, I find
R(D) = 0.2991 -0.0058 +0.0052
R(D*) = 0.2456 -0.0087 +0.0073
Comparing this to @Jorge-Alda's notebook, there seems to be small difference for RD, while RD* agrees very well with the result assuming a Gaussian distribution for the form factors. But in any case, these uncertainties are still much larger than those in https://github.com/flav-io/flavio/issues/156#issuecomment-818825181. At the moment I have no idea where this difference could come from.
The bug has been fixed by @Jorge-Alda in PR #160.
I am still keeping this issue open since the uncertainties I find from https://zenodo.org/record/3376898 are considerably larger than those in 1912.09335 and further investigation might be interesting.
Regarding the different size of the uncertainties, did you use the weights provided in the data set?
Update on this issue: I have now compared the flavio and EOS predictions using the same input values for the HQE parameters in both cases (see this notebook).
Using flavio, I find the same values as quoted above (but now I also used the weights provided in the data set):
R(D) = 0.300 ± 0.005
R(D*) = 0.246 ± 0.008
Using EOS, I exactly reproduce the numbers in eq. (11) in arXiv:1908.09398:
R(D) = 0.298 ± 0.003
R(D*) = 0.247 ± 0.006
So this difference between flavio and EOS is obviously not caused by the HQE parameters. I would assume that it is due to differences in some other input parameters but I don't fully understand it yet.
So this difference between flavio and EOS
Which difference do you refer to? The small difference between the values you quote?
Is there actually still an issue with the current flavio prediction or not?
I would assume that it is due to differences in some other input parameters but I don't fully understand it yet.
First guess: m_c?
m_c is a good guess actually, I remember having a different treatment from Danny's at some point.
So this difference between flavio and EOS
Which difference do you refer to? The small difference between the values you quote?
The uncertainty of RD in flavio is nearly twice as large as in EOS and the central value differs by roughly the size of the EOS uncertainty. For RD* the situation is better but also there the flavio uncertainty is considerably larger. Since in both cases exactly the same HQE parameters have been varied and all other parameters are just set to their central value, I still find this a bit surprising.
Is there actually still an issue with the current flavio prediction or not?
This is not clear to me. Maybe the EOS uncertainty is just smaller because of some other parameter having a different central value than in flavio.
@peterstangl might be good to see one more significant digit in the uncertainty :slightly_smiling_face:
@peterstangl might be good to see one more significant digit in the uncertainty slightly_smiling_face
I have rerun the notebook and get (with 5000 different samples)
R(D) = 0.29967 ± 0.00551 (flavio)
R(D) = 0.29820 ± 0.00348 (EOS)
R(D*) = 0.24556 ± 0.00790 (flavio)
R(D*) = 0.24658 ± 0.00608 (EOS)
So "nearly twice as large" was maybe a bit exaggerated but the uncertainty of RD is still 60% larger in flavio.
I was looking in the code for the place where other parameters enter and found the function get_hqet_parameters.
https://github.com/flav-io/flavio/blob/bc3a254924af702372a1531caf6e16f94e41c02a/flavio/physics/bdecays/formfactors/hqet.py#L10-L24
@DavidMStraub do you remember where these relations are taken from? I think a different definition of these parameters in EOS could be responsible for the different results.
@peterstangl For reference, here are the values we use in EOS:
https://github.com/eos/eos/blob/417d909b35f7eac3b1ac7b6a552ff68aa20ff41d/eos/form-factors/mesonic-hqet.hh#L185-L191
Thanks @DavidMStraub and @dvandyk! I found the relations in section II.D of arXiv:1703.05330 and next I will compare the values of the HQET parameters and their impact on the results in EOS and flavio.
It looks like the main source of the difference between flavio and EOS is the value of mb1S. Other parameters like e.g. lambda_1 also differ in the flavio and EOS implementations, but these differences have a negligible impact.
Setting mb1S in the flavio implementation to the value used by EOS, mb1S=4.71, instead of the value computed by RunDec, mb1S=4.68, reproduces exactly the EOS result within flavio.
The value used in EOS is probably taken from arXiv:1703.05330, eq. (24), which reads
mb1S = 4.71 +- 0.05 GeV
From the code quoted by @dvandyk, https://github.com/eos/eos/blob/417d909b35f7eac3b1ac7b6a552ff68aa20ff41d/eos/form-factors/mesonic-hqet.hh#L185-L191 it looks like EOS only uses the central value and does not take into account any uncertainty on mb1S. Given the sizeable sensitivity of the final result on mb1S, I would expect that taking such uncertainty into account would considerably increase the uncertainty of the EOS result and bring it in line with the result found with flavio.
@dvandyk and @MartinJung1978, can you confirm that the uncertainty of mb1S has not been taken into account when deriving the results in arXiv:1908.09398, eq. (11)?
@peterstangl Thanks for the investigation! @MartinJung1978 and I just had an opportunity to discuss this. To be brief:
- We confirm that we consider mb1S to be a fixed parameter in our HQE fit, and we think that this is correct. flavio should not vary mb1S when using our parameter samples.
- We are not sure that varying mb1S in a fit is a good idea, either. We are happy to discuss this further offline.
Thanks a lot @dvandyk and @MartinJung1978 for discussing this and for your comments!
So I guess to be consistent, the HQET parameters used together with your samples should be the same as those used to generate the samples, i.e. the numbers defined in https://github.com/eos/eos/blob/417d909b35f7eac3b1ac7b6a552ff68aa20ff41d/eos/form-factors/mesonic-hqet.hh#L185-L191
I would still be curious why you do not vary these parameters in your fit and then e.g. provide combined samples for both the HQET parameters and the form factor parameters. I agree it might be easier to discuss this offline. (We can then add the outcome of the discussion to this thread for future reference).