Free 'communicate(Field3D)' function?

[johnomotani]

In discussion under #1470, @d7919 suggested

I wonder if it would be worth introducing a free function communicate(Field3D &var) { var.getMesh()->communicate(var); return;}

I like the idea. Could tidy up user code, reduce uses of global mesh pointer, and allow handling of fields with different meshes in the same call (i.e. if f and g have different fieldmeshes, communicate(f, g) could handle calling communicate from each mesh).

[d7919]

Of course the downside is that if you're communicating lots of fields on the same mesh this approach of communicating one after the other would lead to lots of small messages rather than a single larger message. I think that's a price worth paying (as we would still provide the more efficient approach as well).

[bendudson]

It should be possible to group together fields with the same mesh, maybe using a map containing a vector of fields, with the mesh as the key. Perhaps not very efficient for small numbers of fields, but the common case of 1-4 fields could be handled differently.

[d7919]

If the communicate was done in a non-blocking manner it might actually be possible to loop over the passed fields, starting the communication and then waiting -- this might help overlap work and comms (although my guess is that this wouldn't work very well).

[dschwoerer]

I didn't know there was a RFE for this. I implemented this basically here

What header should this go into? Make mesh.hxx more expensive?

The version I implemented assumes all fields are on the same mesh (because I only needed this) I think it should be easy for the user to call communicate for all meshes separately ...

[johnomotani]

Maybe have an ASSERT2 that the meshes of all fields are the same? I'd think that would be enough...

[ZedThree]

What header should this go into? Make mesh.hxx more expensive?

Maybe physicsmodel.hxx? Are we likely to use it much in the library? Quick search looks like it won't be much of a benefit to library code, I think we mostly already have a local mesh* for other reasons, and we rarely (ever?) communicate multiple fields in library code.

Also, could use some trick like: https://www.fluentcpp.com/2019/03/05/for_each_arg-applying-a-function-to-each-argument-of-a-function-in-cpp/ to just call Mesh::communicate on each argument. It's pretty gross without C++17 though.