lawrenceccheung

Yes, everything is a submodule. My perspective is that something changed over the last two months -- either in the ExaWind code, or Summit hardware, or both -- which is...

These cases I'm running are *without* hypre, so with the amrex defaults. Another data point we just got is that if we run just a simple, single-level precursor: https://github.com/lawrenceccheung/AWAKEN_summit_setup/blob/main/precursor/StableABL1/KingPlains_stable_precursor9.inp then...

Yes, I tried out 4b71037218723e0c63d54c140423ef503ac3c912 which includes Paul's fixes. Just to keep everyone up to date, I talked with @PaulMullowney and @psakievich earlier, and I'm going to get some debug...

More data for those interested in this problem. Here's the verbose output from the nodal projection: ``` Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 0.05129219173 *...

Oh that build that @alhs6577 is from commit 4b71037, and compiled using spack-manager.

In case it helps, I put together a log of different runs I was doing to try and bisect the case. I will keep adding to this list with more...

@asalmgren @PaulMullowney @psakievich it occurred to me that we might have a way to determine if this is a software or a hardware issue. I have an old executable from...

yes, I'm including system software when I say hardware. Although, I haven't changed the way I compile amr-wind in the last 6 months or so -- they should all be...

Another interesting data point. I just tried a run using 9537522, which is based off the most recent branch 4b71037 with additional radar scan functionality. That failed, but the bottom...

Latest run with CPU's only (July 7 with e97f8472 above), also failed, but please note -- it failed (as in core dumped) on the first *MAC projection* step, not on...