WarpX icon indicating copy to clipboard operation
WarpX copied to clipboard

Add call to HandleParticlesAtBoundaries in WarpXEvolve.cpp to prevent oob segfaults

Open archermarx opened this issue 1 year ago • 3 comments

Fixes bug found here https://github.com/ECP-WarpX/WarpX/issues/5065 and partially addresses https://github.com/ECP-WarpX/WarpX/issues/5113

This PR just adds a call to HandleParticlesAtBoundaries to the synchronization step part of WarpXEvolve.cpp to prevent unhelpful memory access errors when particles travel too far in one step.

Before, this error would be thrown:

amrex::Abort::3::CUDA error 700 in file /home/marksta/src/warpx/build/_deps/fetchedamrex-src/Src/Base/AMReX_GpuDevice.cpp
line 648: an illegal memory access was encountered !!!
SIGABRT

Now, no error is thrown and the simulation finishes successfully.

Ideally, we would still like to warn users about CFL violations in ES simulations, or adapt the timestep to account for a user-specified CFL number (which may be greater than 1 for semi-implicit schemes). I'll keep looking at that.

archermarx avatar Aug 14 '24 15:08 archermarx

Looks like the qed_schwinger and Point_of_contact_EB_3d FAILED tests fail, but only by a bit (errors on order of 0.01 relative tolerance vs the requested 0.001). I could wrap this line in #if not defined(WARPX_QED) || not defined (WARPX_EB), perhaps

Also, the dsmc_1d test fails now, with relative errors of O(1e-8) as opposed to 1e-9.

archermarx avatar Aug 14 '24 16:08 archermarx

I think one could reset these checksums. The test changes the order of particles that go into the checksum by redistributing them. Long floating point sums are not commutative.

ax3l avatar Aug 15 '24 22:08 ax3l

How could I go about doing that?

archermarx avatar Aug 16 '24 14:08 archermarx

Closing this for now. May create a fresh PR in the future.

archermarx avatar Sep 18 '24 19:09 archermarx