jwst icon indicating copy to clipboard operation
jwst copied to clipboard

Test with safe casting

Open eslavich opened this issue 3 years ago • 1 comments

Description

This PR is to run the tests with a branch of stdatamodels that raises errors on unsafe array casts. It should not be merged.

Checklist

  • [ ] Tests

  • [ ] Documentation

  • [ ] Change log

  • [ ] Milestone

  • [ ] Label(s)

eslavich avatar Jun 23 '21 21:06 eslavich

So there's a lot of

ValueError: Array dtype float64 cannot be safely cast to schema dtype float32

here. Does this mean we have to do lots of .astype(np.float32) calls (which makes a truncated copy) on arrays before assigning them back to the datamodel attribute that has the schema checking its dtype?

I've always been skeptical of keeping our data arrays as float32 in our datamodel schemas. If we are going to do safe casting checks (which we should do), this will add a lot of by-hand casting when we want to be doing calculations in 64-bit.

I.e. the disconnect of wanting the in-memory representation of all our data arrays to be 64-bit, but when finally written out to a FITS file to be 32-bit (to save storage costs?) is not rearing its head. Again.

And there's already a lot of casting to 64-bit of 32-bit arrays that are read in.

Can we revisit this? What are the trade-offs?

jdavies-st avatar Jun 30 '21 20:06 jdavies-st