dtype-next icon indicating copy to clipboard operation
dtype-next copied to clipboard

dtype/alloc-zeros error

Open benjamin-asdf opened this issue 1 year ago • 4 comments

Hi, I suppose this is a bug:

cnuernber/dtype-next {:mvn/version "10.113"}
(require '[tech.v3.datatype :as dtype])

(dtype/alloc-zeros :boolean 20)
Error printing return value (ClassCastException) at tech.v3.datatype.native_buffer.ObjectNativeBuf/subBuffer (native_buffer.clj:483).
class java.lang.Long cannot be cast to class clojure.lang.IFn (java.lang.Long is in module java.base of loader 'bootstrap'; clojure.lang.IFn is in unnamed module of loader 'app')

works with :int8.

More info:


  (let [v (dtype/alloc-zeros :boolean 2)]
    (dtt/->tensor v))
  
  ;; Execution error (NullPointerException) at tech.v3.datatype.copy/unsafe-copy-memory (copy.clj:70).
  ;; Cannot invoke "java.lang.Number.longValue()" because "x" is null

benjamin-asdf avatar Jun 20 '24 07:06 benjamin-asdf

Yep definitely a bug

cnuernber avatar Jun 23 '24 12:06 cnuernber

Looking into this - alloc-zeros is guaranteed to return a native buffer and native buffers in general do not support boolean types as those are logical types with either 1 bit or 8 bit representations depending on context. The dtype/functional namespace in general does treat 0 as false and anything else as true in keeping with the 'c' semantics.

So to make a long story a bit shorter I could alias allocating an int8 buffer with this and then return a custom reader or something like that - is this what you really want?

cnuernber avatar Sep 12 '24 15:09 cnuernber

My suggestion, what would have helped me in the situation, would be to throw an error for :boolean in alloc zeros that explains this. Then the user can chose int8. maybe: "alloc-zeros: native buffers do not support boolean types. You probably want to use int8 instead."

Yea I agree that the alias idea doesn't sound so great. It's nice that dtype is close to metal like this!

benjamin-asdf avatar Sep 13 '24 09:09 benjamin-asdf

Better error messages - where have I heard that before?? ;-)

Good solution it makes great sense.

cnuernber avatar Sep 13 '24 15:09 cnuernber