Brute-force re-initialization

[uekerman]

This is my long-promised first attempt to elaborate on the big picture on how to realize a brute-force re-initialization. Feel free to extend the issue.
It might turn out more complicated than I thought, especially the triggering of the re-initialization, where we need to carefully weigh the consequences of different solutions.

The story

"As a user of preCICE, I want to adapt my coupling mesh during a simulation, such that I can handle re-meshing of my solver (or load-balancing or dynamically adaptive mesh refinement) without restarting the simulation."

• Re-initialization means that user can adapt the mesh during a coupled simulation, define new mesh primitives, delete old ones
• Brute-force means for the user that only the complete mesh can be marked invalid
• Brute-force means for the developer that we don't keep temporary information, but throw away all data structures and set them up again from scratch. This means we don't worry about performance making this the baseline for further improvements. This includes:
  • Re-partitioning
  • Slaves-slaves communication
  • QN information from previous timesteps

Assumptions for simplification

• Only two coupled participants

Open questions

• Do we allow re-init within an implicit coupling loop or only after a timestep is completed?
  • I currently don't see any additional complexity when allowing this within an implicit loop.
• Do we allow re-init within subcycling?
  • Probably no big technical issue (we could simply use the last changed mesh)

Anyhow: Let's not consider the general case at first, so no subcycling, no changes during implicit coupling

Historical remarks

• This story was possible with the old serial version of preCICE (way back, see Bernhard's thesis)
• It was much simpler back than:
  • No need to update partitions and slaves-slaves communication channels
  • Thus, it was sufficient if only one participant knew about this.

Status quo

What happens currently in initialize() and what we have to re-do for re-initialization.

• establish master-master communication -> we could, but we don't want
• compute partitions -> we have to (attention: also allocates data values)
• establish slave-slave communication -> we have to
• initialize watchpoints
  • find closest vertex -> we have to
  • set up output file -> we cannot
• initialize coupling scheme
  • set start time -> we could, but we don't want
  • set up convergence measures (coupling data gets assigned, but not set up of data structures) -> we could, but we don't want to
  • set up data matrices (reserve memory for old values for c.m. and extrapolation) -> we have to
  • initialize pp (reserve memory, QN columns etc.) -> we have to
  • requireAction(writeIterationCheckpoint) -> we cannot
  • evaluate whether initial data has to sent or received -> we could , but we don't want
  • exchange data if necessary -> we cannot
• map read data if necessary -> we probably don't have to, but let's keep an eye on this
• perform data actions (if necessary after receiving data) -> we probably don't have to, but let's keep an eye on this

Subtasks

• [x] A: implement reinitialize() for acceleration (deletes old data structures creates new ones)
• [ ] B: implement reinitialize() for coupling scheme (deletes old data structures creates new ones)
• [x] C: implement reinitialize() for watchpoint (deletes old data structures creates new ones)
• [x] D: implement mechanism in SolverInterface to steer a re-initialization (deletes mesh and tells other participant)
• [x] E: implement "reinitialize" for SolverInterface (compute partitions, establish slave-slave communication, re-initialize watchpoints, re-initialize coupling scheme)
  • pre-condition: mesh has been filled again (with e.g. setMeshVertices)
  • both participants need to call this
  • additional API function or part of advance?

Related TODOs

• [x] Error messages if mesh primitives are defined when not allowed (e.g. setMeshVertices() is called after initialize() #228

[precice-bot]

This issue has been mentioned on preCICE Forum on Discourse. There might be relevant details there:

https://precice.discourse.group/t/fsi-with-interfoam-and-calculix/277/2