duckdb-rs icon indicating copy to clipboard operation
duckdb-rs copied to clipboard

Appender support for arrays (or alternative)

Open cipriancraciun opened this issue 11 months ago • 5 comments

At the moment, although the Value and ValueRef seem to mention support for arrays, lists, and a few related items, none of them seem to be implemented (i.e. the code panics with unsupported!()) when used in Appender::append_row or even for parameter binding in prepared statements.

It would be nice, since DuckDB does support array / list types, to be able to bulk insert data (or at least insert row-by-row) that contains such types.

In the interim, is there perhaps a workaround? (Besides serializing the data as JSON and using COPY, or writing a prepared statement that deserializes data a string via JSON.)

cipriancraciun avatar Jan 22 '25 14:01 cipriancraciun

I do not believe there is a function in the C api to append an array or list, but it would require the value to be coverted to a data chunk and then appended with duckdb_append_data_chunk

If you have arrow data, I think it is possible to do this in the current rust api with append_record_batch(), which will convert the record batch to a data chunk and then use duckdb_append_data_chunk.

era127 avatar Feb 12 '25 02:02 era127

Looking at the current Rust API (https://docs.rs/duckdb/latest/duckdb/struct.Appender.html?search=batch), I don't see any appender.append_record_batch() method; is this perhaps in a different Rust crate? (I've also checked the libduckdb-sys crate that I believe is the C++ wrapper for Rust.)

cipriancraciun avatar Feb 12 '25 05:02 cipriancraciun

It is in the appender-arrow feature here.

era127 avatar Feb 12 '25 06:02 era127

I think duckdb_append_value was added in 1.2.0 (https://github.com/duckdb/duckdb/pull/15065). I guess that can be used to append arrays and other types.

DorianCzichotzki avatar Feb 13 '25 15:02 DorianCzichotzki

Is support for appender arrays coming to JDBC (org.duckdb.DuckDBAppender)? I only see byte[] implemented.

ajweber avatar May 01 '25 14:05 ajweber

Please use append_record_batch as recommended (available via the appender-arrow feature). The Rust docs will include the method in the next release (1.3.2) thanks to #524.

Please refer to https://github.com/duckdb/duckdb-rs/blob/v1.3.1/crates/duckdb/src/appender/arrow.rs#L30 for the time being.

mlafeldt avatar Jul 08 '25 10:07 mlafeldt