Derek Gaston
Derek Gaston
Damn - I messed up something with metaphysicl. Any ideas @roystgnr and @lindsayad ?
Yep - I had to basically copy all of the metaphysicl headers into `include/metaphysicl` to make my in-tree build work. I guess I then ran the scripts and accidentally added...
Don't merge this yet. I want to do some more testing of it first....
Oh damn. I've been getting bit by this - and I didn't know it! Going to pull this and try the fix now. Damn. Damn. Damn.
Well - this fixes the "hang" - but causes an enormous memory spike that causes me to run out of memory. Still investigating...
@roystgnr am I insane - or does this push all boundary nodes to _all_processors_!? That doesn't work with 200M elements on 5k procs... Unfortunately - we are severely handicapped here...
Ok - I see the algorithm now - it doesn't push to all processors - it pushes to owners and then pulls ghosts. Seems reasonable... I'm going to try to...
I put print statements... and sleep statements all through here... and then _it passed_! Sigh - may be a Heisenbug... and it might have to do with my own NBX...
Currently stuck on an Allgather in `DofMap::distribute_dofs()` BTW....
It eventually made it past that... but that AllGather is seriously expensive at 4k procs... BTW: Reverting from the NBX algorithm seemed to help - but the code is now...