PETSc Example Int64 CI

[jeremylt]

We can now use the PETSC_ARCH=mpich-hip-int64 in CI. @jrwrigh we probably don't need to run every fluids test in int64. Do you have a minimal subset you'd like to see? (we should probably trim out some of the GPU runs too since that's the bulk of our CI time) If you have a list of test cases that should be run, I can gen up a PR. Its easiest if the tests don't involve any file IO

[jrwrigh]

Any reason to not convert one of the existing test sets(? Idk the exact terminology here) to use 64bit? Then leave the rest as 32?

That's probably what I'm going to do locally; always test with 64-bit.

Edit: Assuming the other PETSc examples can use 64 bit ints.

[jeremylt]

The default behavior would be the same as we currently have for CPU vs GPU - repeat the entire list of tests on navierstokes.c.

For every test that we don't need to run in int64, we'd change //TESTARGS(name="Advection 2D, rotation, explicit, su, consistent mass") to //TESTARGS(name="Advection 2D, rotation, explicit, su, consistent mass",only="int32")

Similarly, we could cut down the GPU test time via //TESTARGS(name="Advection 2D, rotation, explicit, su, consistent mass",only="cpu")

These can stack too //TESTARGS(name="Advection 2D, rotation, explicit, su, consistent mass",only="cpu,int32")

[jrwrigh]

What I meant was is there any harm in changing https://github.com/CEED/libCEED/blob/43dcc2defac7c7180945fbd82b0ef1131721d2ad/.gitlab-ci.yml#L98 to use mpich-hip-int64? That just seems like the simpler solution, assuming the infrastructure for only="cpu,int32" isn't already setup.

[jeremylt]

assuming the infrastructure for only="cpu,int32" isn't already setup.

Its already there

[jeremylt]

• export PETSC_DIR=/projects/petsc PETSC_ARCH=mpich-hip && git -C $PETSC_DIR -c safe.directory=$PETSC_DIR describe

doing this would require changing a lot of output in examples/fluids/tests-output. Does every single fluids test now require a .bin one way or another?!?

[jedbrown]

I don't understand this issue. We can now read the output files regardless of which ints are used, or we can do tests that don't have reference output.

[jeremylt]

Maybe I misunderstood how that PR worked?

[jedbrown]

In my PETSc MR? It just works. You can write from either i32 or i64 and read from either. We don't have to change data files or code.

[jrwrigh]

Yeah, it should Just Work:tm: without any changes. I've been doing my local tests for the past few days using the 64bit PETSc without changing any of the outputs and it's running fine.

[jrwrigh]

Does every single fluids test now require a .bin one way or another?!?

Yes... if that should change, let me know.

[jeremylt]

The bins are a fair amount of data which is why I've been trying to use them only where needed in Ratel

[jrwrigh]

Gotcha. I try and keep the file sizes quite minimal. Currently, the total size of the bins is less than all the qfunctions header files.