StrideArraysCore.jl icon indicating copy to clipboard operation
StrideArraysCore.jl copied to clipboard

throw error or support non bitstype?

Open Roger-luo opened this issue 4 years ago • 6 comments

currently for example BigFloat will crash

StrideArray{BigFloat}(undef, (4, 4))

Roger-luo avatar Apr 04 '21 02:04 Roger-luo

Hmm, vectors seem to work

julia> using StrideArraysCore

julia> x = ["a", "b", "hello", "world"]
4-element Vector{String}:
 "a"
 "b"
 "hello"
 "world"

julia> StrideArray(x)
4-element StrideArray{Tuple{Int64}, (true,), String, 1, 1, 0, (1,), Tuple{StaticInt{8}}, Tuple{StaticInt{1}}, Vector{String}} with indices static(1):4:
 "a"
 "b"
 "hello"
 "world"

julia> y = [("x", 3), ("y", 8)]
2-element Vector{Tuple{String, Int64}}:
 ("x", 3)
 ("y", 8)

julia> StrideArray(y)
2-element StrideArray{Tuple{Int64}, (true,), Tuple{String, Int64}, 1, 1, 0, (1,), Tuple{StaticInt{16}}, Tuple{StaticInt{1}}, Vector{Tuple{String, Int64}}} with indices static(1):2:
 ("x", 3)
 ("y", 8)

julia> z = [rand(2), rand(4)]
2-element Vector{Vector{Float64}}:
 [0.10740915258506245, 0.37583336292320535]
 [0.8595843553115332, 0.4806825510809891, 0.6416522729462721, 0.388031325818297]

julia> StrideArray(z)
2-element StrideArray{Tuple{Int64}, (true,), Vector{Float64}, 1, 1, 0, (1,), Tuple{StaticInt{8}}, Tuple{StaticInt{1}}, Vector{Vector{Float64}}} with indices static(1):2:
 [0.10740915258506245, 0.37583336292320535]
 [0.8595843553115332, 0.4806825510809891, 0.6416522729462721, 0.388031325818297]

julia> a = rand(BigFloat, 2)
2-element Vector{BigFloat}:
 0.04156627097611492688430582243643709582802351551281496652702285425218285789243257
 0.2413829526464971427820962585619924942902587484891776557325573322984283151913453

julia> StrideArray(a)
2-element StrideArray{Tuple{Int64}, (true,), BigFloat, 1, 1, 0, (1,), Tuple{StaticInt{8}}, Tuple{StaticInt{1}}, Vector{BigFloat}} with indices static(1):2:
 0.04156627097611492688430582243643709582802351551281496652702285425218285789243257
 0.2413829526464971427820962585619924942902587484891776557325573322984283151913453

There's a bug in cartesian indexing.

chriselrod avatar Apr 04 '21 03:04 chriselrod

julia> x
5×10 Matrix{BigFloat}:
 0.650397  0.228057  0.295901  0.961686  0.658904   0.234168  0.00907257  0.115974  0.221669  0.016437
 0.170399  0.947094  0.010932  0.707836  0.738352   0.890069  0.295456    0.859759  0.31087   0.771494
 0.861343  0.729251  0.181878  0.247554  0.0752045  0.29782   0.402837    0.50297   0.496431  0.73992
 0.1018    0.596116  0.721762  0.251329  0.818825   0.661841  0.138965    0.500957  0.130641  0.833923
 0.43165   0.262839  0.710115  0.805982  0.773598   0.270191  0.412482    0.806387  0.579312  0.236581

julia> StrideArray(view(x, :, 1:2))
5×2 StrideArray{Tuple{Int64, Int64}, (true, true), BigFloat, 2, 1, 0, (1, 2), Tuple{StaticInt{8}, Int64}, Tuple{StaticInt{1}, StaticInt{1}}, SubArray{BigFloat, 2, Matrix{BigFloat}, Tuple{Base.Slice{Base.OneTo{Int64}}, UnitRange{Int64}}, true}} with indices static(1):5×static(1):2:
 0.650397  0.234168
 0.170399  0.890069
 0.861343  0.29782
 0.1018    0.661841
 0.43165   0.270191

julia> x[:,[1,6]]
5×2 Matrix{BigFloat}:
 0.650397  0.234168
 0.170399  0.890069
 0.861343  0.29782
 0.1018    0.661841
 0.43165   0.270191

julia> sizeof(BigFloat) / sizeof(Int)
5.0

Part of the code still assumes Base.allocatedinline, so for BigFloat it assumes 5x the correct offset as the stride between columns.

chriselrod avatar Apr 04 '21 03:04 chriselrod

It should work given VectorizationBase 0.19.26, with the caveat that it won't null-initialize or null-check on getindex.

chriselrod avatar Apr 04 '21 08:04 chriselrod

I'm currently on

(BQCESubroutine) pkg> st
     Project BQCESubroutine v0.1.0
      Status `~/code/julia/BQCESubroutine/Project.toml`
  [4fba245c] ArrayInterface v3.1.6
  [b630d9fa] CheapThreads v0.1.11 `https://github.com/JuliaSIMD/CheapThreads.jl.git#master`
  [e2ba6199] ExprTools v0.1.3
  [bdcacae8] LoopVectorization v0.12.11
  [d8e11817] MLStyle v0.4.9
  [bac558e1] OrderedCollections v1.4.0
  [d1fa6d79] StrideArrays v0.1.5
  [8290d209] ThreadingUtilities v0.4.1
  [3d5dd08c] VectorizationBase v0.19.26
  [66df03fb] YaoLocations v0.1.3
  [37e2e46d] LinearAlgebra

I still get segment faults it seems due to the printing

julia> using StrideArrays

julia> a = StrideArray{BigFloat}(undef, (4, 4));

julia> a
4×4 StrideArray{Tuple{Int64, Int64}, (true, true), BigFloat, 2, 1, 0, (1, 2), Tuple{StaticInt{40}, Int64}, Tuple{StaticInt{1}, StaticInt{1}}, Vector{BigFloat}} with indices static(1):4×static(1):4:

signal (11): Segmentation fault
in expression starting at none:0
sig_match_simple at /buildworker/worker/package_linux64/build/src/typemap.c:189 [inlined]
jl_typemap_entry_assoc_exact at /buildworker/worker/package_linux64/build/src/typemap.c:954
jl_typemap_assoc_exact at /buildworker/worker/package_linux64/build/src/julia_internal.h:1189 [inlined]
jl_lookup_generic_ at /buildworker/worker/package_linux64/build/src/gf.c:2359 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2415
#sprint#385 at ./strings/io.jl:103
unknown function (ip: 0x7f1f501a1cc4)
sprint##kw at ./strings/io.jl:101 [inlined]
alignment at ./show.jl:2582
alignment at ./arrayshow.jl:68
unknown function (ip: 0x7f1f501a1a66)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
_print_matrix at ./arrayshow.jl:201
unknown function (ip: 0x7f1f501a0d9a)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1703 [inlined]
jl_f__call_latest at /buildworker/worker/package_linux64/build/src/builtins.c:714
#invokelatest#2 at ./essentials.jl:708 [inlined]
invokelatest at ./essentials.jl:706 [inlined]
print_matrix at ./arrayshow.jl:170 [inlined]
print_matrix at ./arrayshow.jl:170 [inlined]
print_array at ./arrayshow.jl:327 [inlined]
show at ./arrayshow.jl:368
unknown function (ip: 0x7f1f5019afac)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
#38 at /buildworker/worker/package_linux64/build/us

Roger-luo avatar Apr 04 '21 18:04 Roger-luo

Making a couple more fixes, but --- --- what's the behavior you want here?

Always check for a null reference on load? Or perhaps only when bounds checking is enabled (assuming it is enabled while printing; if not, this isn't an option?). I'll then also have to make sure it null-initializes them even when using MemoryBuffer instead of a Vector for the backing memory.

chriselrod avatar Apr 04 '21 19:04 chriselrod

I think it's OK if it is constructed with null, but perhaps handle it like how Julia handles #undef regions in Array? you can't access that piece, but it will be printed as #undef (in this case #null?)

(I mostly just don't want it crash but to get an error if it doesn't work, so any behaviour doesn't crash sounds good to me ;-) )

Roger-luo avatar Apr 04 '21 23:04 Roger-luo