JLD2.jl icon indicating copy to clipboard operation
JLD2.jl copied to clipboard

Review use of `unsafe_wrap`

Open nhz2 opened this issue 3 months ago • 3 comments

unsafe_wrap is very difficult to use correctly due to how type-based aliasing analysis works in Julia; some of the uses in this package seem suspect. It might be useful to define a PtrArray type for internal use here, like the one in PtrArrays.jl, as an alternative.

nhz2 avatar Sep 08 '25 14:09 nhz2

I believe I am unaware of that potential problem. Do you have any resource on this?

JonasIsensee avatar Sep 08 '25 14:09 JonasIsensee

The only official thing is the docstring of Base.unsafe_wrap

But from discussions on Slack, it is rarely safe to use on a pointer to memory managed by Julia.

nhz2 avatar Sep 08 '25 15:09 nhz2

Indeed, this section sounds worrying.

the programmer is responsible also for ensuring that the underlying data is not accessed through two arrays of different element type, similar to the strict aliasing rule in C.

The strict aliasing rule is certainly violated since that is the whole point. Usage of the pointers in the compression pipeline is, of course, fully sequential:

  • Construct unitialized Array of the target type
  • unsafe_wrap as byte buffer
  • fill with byte data read from file
  • return original array filled with data

Usage for treating strings as byte arrays is probably unnecessarry. That could be changed.

JonasIsensee avatar Sep 08 '25 15:09 JonasIsensee