duckdb-rs

duckdb-rs copied to clipboard

Hello

I had no Err() when using append_row(params![None])) when the target column is indeed NOT NULL. And did not find a way to check how many lines were actually inserted ...

Does it sound like something fishy here or was it the exepected/normal behavior ?

Got it in duckdb-rs 0.8 but could not really reproduce with latest version cause it has other issues (Assertion 'types == input.GetTypes()' failed panicking from cpp side)

duckdb = { version = "0.8", features = [
  "bundled",
  "modern-full",
  "parquet",
  "json",
  "r2d2",
] }
libduckdb-sys = { version = "0.8", features = ["bundled", "parquet", "json"] }

[[package]]
name = "duckdb"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "393c174261032fbed2d1a67ae78fd3aa5e9a42687a23bb105b2a00a646a36dda"

[[package]]
name = "libduckdb-sys"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "84f1e0bc90f97fb7fe64f5d1d8450f4eef8db725efcf7a29a30b7fdfd2705d81"

@jrouaix I agree with your suspicion. I've used the appender to insert 650_000 simulated persons into duckdb, and noticed a couple of things:

• Loading was VERY slow (compared to sqlite)
• After calling appender's flush method, the data should have been written to the database. However, it was not (as confirmed by a select count query on the table).
• When inserting data using sqlite, the unique ID constraint was violated several times. Either appender was too slow, or it never reported it (or the underlying C implementation never checked it).

So we didn't yet updated further than 8.0, but perhaps version 10 is freed from this bug. Will try ... one day soon 🤞🏼

@jrouaix Unfortunately, I have tried this with the latest version, 0.10.1.

The appender api should error as documented in this issue

However it should not be much slower than SQLite, is it possible to share a mwe.

I found if you are using the arrow version, try flushing it every <2048 len elements to work.