Distances.jl icon indicating copy to clipboard operation
Distances.jl copied to clipboard

conversion to pointer not defined for `SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}`

Open ForceBru opened this issue 2 years ago • 1 comments

Below is a copy of the issue I opened in the Clustering.jl repo. In the comments of that issue it was suggested that I open an issue here, so here it is.


I'm trying to run clustering on a view with indices of type UInt64, but getting the error in the title.

Code:

import Pkg
Pkg.activate(temp=true)
Pkg.add("Clustering", io=devnull)
Pkg.status()

import Clustering

data = rand(100)
my_view = @view data[1:UInt64(50)]

Clustering.kmeans(reshape(my_view, 1, :), 2)

Output:

~/test $ julia uint64_view.jl
  Activating new environment at `/var/folders/ys/3h0gnqns4b98zb66_vl_m35m0000gn/T/jl_TpqK3f/Project.toml`
      Status `/private/var/folders/ys/3h0gnqns4b98zb66_vl_m35m0000gn/T/jl_TpqK3f/Project.toml`
  [aaaa29a8] Clustering v0.14.2
ERROR: LoadError: conversion to pointer not defined for SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] unsafe_convert(#unused#::Type{Ptr{Float64}}, a::SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true})
    @ Base ./pointer.jl:67
  [3] unsafe_convert(#unused#::Type{Ptr{Float64}}, a::Base.ReshapedArray{Float64, 2, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}, Tuple{}})
    @ Base ./reshapedarray.jl:281
  [4] gemm!(transA::Char, transB::Char, alpha::Float64, A::Matrix{Float64}, B::Base.ReshapedArray{Float64, 2, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}, Tuple{}}, beta::Float64, C::Matrix{Float64})
    @ LinearAlgebra.BLAS /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/LinearAlgebra/src/blas.jl:1458
  [5] gemm_wrapper!(C::Matrix{Float64}, tA::Char, tB::Char, A::Matrix{Float64}, B::Base.ReshapedArray{Float64, 2, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}, Tuple{}}, _add::LinearAlgebra.MulAddMul{true, true, Bool, Bool})
    @ LinearAlgebra /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/LinearAlgebra/src/matmul.jl:671
  [6] mul!
    @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/LinearAlgebra/src/matmul.jl:377 [inlined]
  [7] mul!
    @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/LinearAlgebra/src/matmul.jl:444 [inlined]
  [8] mul!
    @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/LinearAlgebra/src/matmul.jl:275 [inlined]
  [9] _pairwise!(r::Matrix{Float64}, dist::Distances.SqEuclidean, a::Matrix{Float64}, b::Base.ReshapedArray{Float64, 2, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}, Tuple{}})
    @ Distances ~/.julia/packages/Distances/gnt89/src/metrics.jl:616
 [10] pairwise!(r::Matrix{Float64}, metric::Distances.SqEuclidean, a::Matrix{Float64}, b::Base.ReshapedArray{Float64, 2, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}, Tuple{}}; dims::Int64)
    @ Distances ~/.julia/packages/Distances/gnt89/src/generic.jl:271
 [11] pairwise(metric::Distances.SqEuclidean, a::Matrix{Float64}, b::Base.ReshapedArray{Float64, 2, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}, Tuple{}}; dims::Int64)
    @ Distances ~/.julia/packages/Distances/gnt89/src/generic.jl:321
 [12] _kmeans!(X::Base.ReshapedArray{Float64, 2, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}, Tuple{}}, weights::Nothing, centers::Matrix{Float64}, maxiter::Int64, tol::Float64, displevel::Int64, distance::Distances.SqEuclidean)
    @ Clustering ~/.julia/packages/Clustering/tt9vc/src/kmeans.jl:138
 [13] kmeans!(X::Base.ReshapedArray{Float64, 2, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}, Tuple{}}, centers::Matrix{Float64}; weights::Nothing, maxiter::Int64, tol::Float64, display::Symbol, distance::Distances.SqEuclidean)
    @ Clustering ~/.julia/packages/Clustering/tt9vc/src/kmeans.jl:70
 [14] kmeans(X::Base.ReshapedArray{Float64, 2, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}, Tuple{}}, k::Int64; weights::Nothing, init::Symbol, maxiter::Int64, tol::Float64, display::Symbol, distance::Distances.SqEuclidean)
    @ Clustering ~/.julia/packages/Clustering/tt9vc/src/kmeans.jl:112
 [15] kmeans(X::Base.ReshapedArray{Float64, 2, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}, Tuple{}}, k::Int64)
    @ Clustering ~/.julia/packages/Clustering/tt9vc/src/kmeans.jl:103
 [16] top-level scope
    @ ~/test/uint64_view.jl:11
in expression starting at /Users/forcebru/test/uint64_view.jl:11
~/test [1] $

/Users/forcebru/test/uint64_view.jl:11 is the last line of the code above.


Now the important part is that this error doesn't appear when I use regular signed integers when indexing. Changing @view data[1:UInt64(50)] to @view data[1:50] "solves" this error. I find it very strange that the type of the index has an effect on the processing of the view. I understand that the view with the UInt64 index is of type SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{UInt64}}, true}, which involves UInt64, while an "ordinary" view like @view data[1:50] is of type SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, now with Int64. So, the type of the index is reflected in the type of the view, which makes sense.

What I don't understand is why the view with the UInt64 causes the error, while the other view with Int64 doesn't. As far as I understand, both views point to the same memory location and have essentially the same structure, except for that UInt64 vs Int64 difference. Looks like some function implementation is missing for UInt64. So this error looks like a bug to me.


julia> versioninfo()
Julia Version 1.6.1
Commit 6aaedecc44 (2021-04-23 05:59 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.7.0)
  CPU: Intel(R) Core(TM) i5-3330S CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, ivybridge)

ForceBru avatar Aug 04 '21 20:08 ForceBru

The error happens in Julia itself, so I've filed an issue one more level up: https://github.com/JuliaLang/julia/issues/42120

nalimilan avatar Sep 04 '21 15:09 nalimilan