Noisy PETSc error messages from `PetscVector`

[ZedThree]

PETSc now complains about us mixing calls to VecGetValues and VecSetValues while building the vectors in PetscVector. This doesn't seem to affect the result, but it is incredibly noisy -- one message per element in the vector. Interestingly it doesn't seem to complain about doing the same for PetscMatrix.

We could avoid the calls to VecGetValues in the PetscVector::Element ctor, but this would break PetscVector::Element::operator+= -- I've not checked if we actually use that anywhere.

One way we could fix this is to instead insert the values into a Field3D/2D while building the vector, and then call VecSetValues once when assembling the vector. I think this should be fine -- the vector is always the same size or smaller than the field? It may even be faster as we'll only have a single call to VecSetValues -- and it may be even possible to just pass the pointer to the field's data? I support we don't even need to use a field, we could use Array (or the vector/matrix types we have built on top of that)?

[johnomotani]

I have a feeling Aaron Fisher implemented something similar to what you're proposing for the Hypre interface (motivated by performance imrovements on GPU). Maybe worth seeing if there's anything copyable from there?

[ZedThree]

Good memory! Yes, although it looks like he had to use raw pointers in order to use cudaMallocManaged to allocate memory on the GPU. I guess we don't have that limitation with PETSc