hp3d icon indicating copy to clipboard operation
hp3d copied to clipboard

Blocking directives in mesh migration

Open jbadger95 opened this issue 1 year ago • 1 comments

Mesh migration can be rather slow on large distributed meshes because each node is sent and received individually with blocking directives. I don't have a large file at the moment but on a moderate-size acoustics problem run on 64 Frontera nodes the timing was:

 migration time:      4.14451 seconds 

This becomes increasingly expensive on large meshes.

Indirect communication and/or batching may be needed.

jbadger95 avatar Mar 10 '23 04:03 jbadger95

As an additional comment: at the moment, a lot of node data (except for middle nodes) are sent/received redundantly; there are various ways of avoiding this which should help with the performance of the data migration.

stefanhenneking avatar Mar 11 '23 23:03 stefanhenneking