Peter Boyle

Hmm.. Test_hmc_EOWilsonCloverFermionGauge: /nfshome/kanamori/work/Grid_tmp/Grid_sycl/Grid/qcd/action/pseudofermion/TwoFlavourEvenOdd.h:171: void Grid::TwoFlavourEvenOddPseudoFermionAction::deriv(const GaugeField&, Grid::TwoFlavourEvenOddPseudoFermionAction::GaugeField&) [with Impl = Grid::WilsonImpl; Grid::TwoFlavourEvenOddPseudoFermionAction::GaugeField = Grid::Lattice]: Assertion `FermOp.ConstEE() == 1' failed. Is a dumb ass logic bomb. Should never happen - Clover...

Might make sense for us to focus on the Force unit tests first - these pass for me and fail for you

Final dH from [email protected]$ ./Test_hmc_EODWFRatio Grid : Message : 1852.775293 s : Total H after trajectory = 521482.769108906 dH = -0.00109183281892911

NB... this was on the --enable-unified=no compile - will take a look now on the --enable-unified=yes. Don't expect any differences - UVM is a LOT easier to get right than...

Interesting -- on Summit I have a fail of Test_hmc_EODWFRatio under UVM compile, while it worked under non-UVM compile. Curious.

Hi Isaaku, yes - I have a fail with Test_hmc_EODWFRatio. The dH is O(130) in the bad run, and O(0.2) in the good run. It's clearly associated with the DWF...

Good news Isaaku - I have a clear and simple smoking gun now in a force test.

Hi, I now know what the problem was and how to fix. Will take me rest of today to get it fixed committed and tested. More info later. Peter

Hi Isaaku, I think it is fixed now - please try again. Peter

I had made the hopping term kernel call for a single direction be non-blocking. This affected only the GPU and specifically only UVM because the non-UVM did a blocking transfer...