Difference estimation ANGSD / ngsF

[jbruxaux]

Hi!

I tried to estimate the inbreeding coefficient for different populations using both ANGSD -doHWE option and ngsF. I use the exact same command to generate the hwe.gz and the glf.gz files (including -SNP_pval to keep only sites that are variable in my pop). I average over the different sites for the ANGSD results, and over the different individuals for the ngsF results. And I get different results... Here an example for a few populations:

ANGSD doHWE	ngsF
0.020	0.047
0.029	0.050
0.028	0.053
0.018	0.054
0.026	0.079
0.028	0.067
0.002	0.044

The difference is maybe not so large (the ngsF values are twice as large, but still very close to 0), but how can we explain it? Is there one method that would be more reliable than the other in this case?

Thanks for your comments!

[fgvieira]

With angsd -doHWE you get an average of per-site inbreeding estimates, while ngsF calculates the average inbreeding across the whole genome. For low coverage samples I'd expect ngsF to be more accurate, but with high coverage both should give similar results.

That said, all your samples have very low inbreeding, so the differences you are seeing might not be relevant. It would be nice to see the differences on a inbred sample.