adbc icon indicating copy to clipboard operation
adbc copied to clipboard

support for decimals dtype in adbc (duckdb tested)

Open matreyes opened this issue 1 year ago • 5 comments

Hi there!, I'm having an issue because duckdb defaults to decimal instead float:

:ok = Adbc.download_driver!(:duckdb)
{:ok, db} = Kino.start_child({Adbc.Database, driver: :duckdb})
{:ok, conn} = Kino.start_child({Adbc.Connection, database: db})
Adbc.Connection.query(conn, "SELECT 11.1111;")

>  {:error, %ArgumentError{message: "not yet implemented for format: `d:6,4`"}}

Sadly I couldn't replicate in sqlite, it seems it always uses float (even casting to decimal or numeric).

Thanks!

matreyes avatar May 15 '24 11:05 matreyes

Hi @matreyes, thanks for reporting this issue. I’ll try to implement this today. :)

cocoa-xu avatar May 15 '24 12:05 cocoa-xu

Perhaps it is worth bringing the Decimal library as a dependency for this one?

josevalim avatar May 15 '24 12:05 josevalim

Perhaps it is worth bringing the Decimal library as a dependency for this one?

I agree. It'd be much easier to start from scratch or casting it to other types

cocoa-xu avatar May 15 '24 12:05 cocoa-xu

Perhaps it is worth bringing the Decimal library as a dependency for this one?

Wouldn't generate a compatibility issue with explorer?

(I was trying with Explorer.Series.from_list([Decimal.new(1, 42, -1)]) and it fails)

matreyes avatar May 15 '24 14:05 matreyes

Explorer does not support a decimal yet but we want to. There is an open issue.

josevalim avatar May 15 '24 15:05 josevalim

Looks like this was recently^1 fixed^2 in adbc

image

Hopefully we can get an adbc 0.3.2 soon! :clap: :clap:

jc00ke avatar May 22 '24 22:05 jc00ke

Hi @jc00ke, my plan is to implement all list types plus the struct one in Adbc.Column in this week (and more types if I have enough time), and then perhaps we can release a new version.

But I'm not sure if @josevalim has any plan for the next release, like if we're only going to release one once all types are supported in Adbc.Column. Below is the list of types that haven't implemented yet:

  • NANOARROW_TYPE_LIST
  • NANOARROW_TYPE_FIXED_SIZE_LIST
  • NANOARROW_TYPE_LARGE_LIST
  • NANOARROW_TYPE_STRUCT
  • NANOARROW_TYPE_SPARSE_UNION
  • NANOARROW_TYPE_DENSE_UNION
  • NANOARROW_TYPE_DICTIONARY
  • NANOARROW_TYPE_MAP
  • NANOARROW_TYPE_EXTENSION

cocoa-xu avatar May 22 '24 22:05 cocoa-xu