Particle-grid periodic communication

[streeve]

This adds the first direct particle-grid capability in Cabana within core, with dependency on Cajita. ~~This is all in particle_grid/ for now, but I'm open to any reorganization and renaming. CabanaPG directly relies on Core and Cajita and is enabled by default if Cajita is.~~

This wraps calls to Distributor, migrate, Halo, and gather to simplify things for any user with periodic boundaries or load balancing needs.

New options include:

• [x] gridDistributor (stolen from ExaMPM), determining new ranks destinations and wrapping through periodic boundaries
• [x] gridMigrate using gridDistributor and calling migrate
• [x] migrate tests
• [x] gridHalo determining particles to be ghosted and any necessary periodic shifts
• [x] ~~PeriodicHalo inheriting from Halo with functor to add periodic shifts~~
• [x] PeriodicShift functor to modify buffer (cannot inherit from Halo because CommPlan contains std member variables, an issue for NVCC)
• [x] gridGather using PeriodicShift and gridHalo and calling gather
• [x] Update to gather for modifying the send buffer with periodic shifts
• [x] halo tests

Closes #77

[streeve]

I think I addressed everything except for how to potentially generalize for other particle-grid methods. Now that I did it as you suggested, this makes much more sense than a separate particle-grid section

[junghans]

@streeve that helped with the build, but now the Cabana_ParticleGridCommunication_test_OPENMP_1 test fails.

[streeve]

@streeve that helped with the build, but now the Cabana_ParticleGridCommunication_test_OPENMP_1 test fails.

Thanks - that's expected at the moment. gridMigrate passes everything, but gridGather has some more issues to fix.

[streeve]

This will fail until rebased with 295 in master; after that, probably a few more fixes to pass halo tests and it will be ready for review.

[sslattery]

#295 has been merged - let's proceed with this one and see if it resolves our comm seg faults.

[sslattery]

Let's rebase this one to get the latest changes for the number of testing ranks and see what the CI does. We likely still have a UVM error though.

[codecov[bot]]

Codecov Report

Merging #263 (0a67b77) into master (d46804f) will decrease coverage by 0.2%. The diff coverage is 94.9%.

@@           Coverage Diff            @@
##           master    #263     +/-   ##
========================================
- Coverage    93.5%   93.3%   -0.3%     
========================================
  Files          37      38      +1     
  Lines        2797    2842     +45     
========================================
+ Hits         2617    2653     +36     
- Misses        180     189      +9     

Flag	Coverage Δ
clang	93.3% <94.9%> (-0.3%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
core/src/Cabana_CommunicationPlan.hpp	0.0% <0.0%> (-98.3%)	:arrow_down:
core/src/Cabana_Halo.hpp	96.1% <95.6%> (+0.3%)	:arrow_up:
core/src/Cabana_ParticleGridCommunication.hpp	99.1% <99.1%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update d46804f...afb82ac. Read the comment docs.

[streeve]

@sslattery this is ready for another review. @junghans codecov is telling me I reduced coverage for CommPlan to zero - any ideas?