CarpetX icon indicating copy to clipboard operation
CarpetX copied to clipboard

add support for different data types in arrays, also on GPU

Open rhaas80 opened this issue 1 year ago • 3 comments

This adds proper support for data types other than double to CarpetX, also for GPUs.

It introduces a helper class AnyVectorthat can hold any data type (similar to gdata in CarpetLib), as well as a number of switch statements to handle openPMD's typed output routines. 

I am told that very new versions of openPMD-api lets one call the internal un-typed output routine, but at the time this code was written, that was not yet possible, and thus may not be possible on all cluster installed copies of openPMD.

Python code to dump data to compare:

#!/usr/bin/env python3

import openpmd_api as io
import sys
import numpy

series = io.Series(sys.argv[1], io.Access.read_only)

print("Read a Series with openPMD standard version %s" %
      series.openPMD)

print("The Series contains {0} iterations:".format(len(series.iterations)))
for i in series.iterations:
    print("\t {0}".format(i))
print("")

it = 0
i = series.iterations[it]
print("Iteration {0} contains {1} meshes:".format(it, len(i.meshes)))
for m in i.meshes:
    print("\t {0}".format(m))
print("")

gf = i.meshes["testoutput_gf_patch00_lev00"]["testoutput_gf"]
shape = gf.shape
print("Field gf has shape {0} and datatype {1}".format(
      shape, gf.dtype))
arrdata = gf.load_chunk()
series.flush()
numpy.savetxt("testoutput_gf.asc", arrdata.flatten())

for var in ["sc", "a1", "a2", "a3"]:
    for suffix in [""]: #, "_int", "_complex"]:
        data = i.meshes["testoutput_"+var+suffix]["testoutput_"+var+suffix]
        print("Field {0} has shape {1} and datatype {2}".format("testoutput_"+var+suffix, data.shape, data.dtype))
        arrdata = data.load_chunk()
        series.flush()
        numpy.savetxt("testoutput_"+var+suffix+".asc", arrdata.flatten())

rhaas80 avatar Dec 06 '24 14:12 rhaas80

Ticket is here: https://bitbucket.org/einsteintoolkit/tickets/issues/2836/add-support-for-different-data-types-to

rhaas80 avatar Dec 06 '24 14:12 rhaas80

Fixing the various CUDA issues right now.

rhaas80 avatar Dec 06 '24 16:12 rhaas80

ping

rhaas80 avatar Feb 22 '25 00:02 rhaas80

anyone interested in this?

rhaas80 avatar Jul 03 '25 16:07 rhaas80

I am definitely interested in this feature. Currently I have to comment out the assert statement in driver.hxx (I think) to make use of any INTs.

shankar-1729 avatar Jul 03 '25 17:07 shankar-1729