rill icon indicating copy to clipboard operation
rill copied to clipboard

Upgrade to duckDB 0.10

Open nishantmonu51 opened this issue 1 year ago • 3 comments

nishantmonu51 avatar Feb 13 '24 09:02 nishantmonu51

Raised a PR to upgrade duckdb version in duckdb driver

esevastyanov avatar Feb 14 '24 12:02 esevastyanov

We need to consider prior breaking changes and a fix according to the comment

// using a patch with changes to support decimal256
// revert to original SDK on next duckDB release
// the go-duckdb v1.5.6 has breaking changes with respect to appender API. The rilldata fork adds query cancellation changes on top of v1.5.5
// details on breaking changes here : https://github.com/marcboeker/go-duckdb/pull/147#issuecomment-1895771714
// changes done on top of v1.5.5 : https://github.com/rilldata/go-duckdb/compare/branch_v1.5.5...rilldata:go-duckdb:appender_revert
replace github.com/marcboeker/go-duckdb v1.5.5 => github.com/rilldata/go-duckdb v0.0.0-20240205054225-f83b5ddead91

esevastyanov avatar Feb 14 '24 12:02 esevastyanov

Not possible to use main as a catalog name

% duckdb main.db
v0.10.0 20b1486d11
Enter ".help" for usage hints.
D show tables;
Error: Binder Error: Ambiguous reference to catalog or schema "main" - use a fully qualified path like "temp.main"

% duckdb meta.db
v0.10.0 20b1486d11
Enter ".help" for usage hints.
D show tables;
┌─────────┐
│  name   │
│ varchar │
├─────────┤
│ 0 rows  │
└─────────┘

esevastyanov avatar Feb 19 '24 15:02 esevastyanov

New DuckDB version is a bit unstable :)

Image

esevastyanov avatar Mar 01 '24 10:03 esevastyanov

^ the issue is a know one for the community

esevastyanov avatar Mar 04 '24 13:03 esevastyanov

Appender fails because there are bunch of unsupported types listed in go-duckdb

var unsupportedTypeMap = map[C.duckdb_type]string{
	C.DUCKDB_TYPE_INVALID:      "INVALID",
	C.DUCKDB_TYPE_DATE:         "DATE",
	C.DUCKDB_TYPE_TIME:         "TIME",
	C.DUCKDB_TYPE_INTERVAL:     "INTERVAL",
	C.DUCKDB_TYPE_HUGEINT:      "HUGEINT",
	C.DUCKDB_TYPE_UHUGEINT:     "UHUGEINT",
	C.DUCKDB_TYPE_DECIMAL:      "DECIMAL",
	C.DUCKDB_TYPE_TIMESTAMP_S:  "TIMESTAMP_S",
	C.DUCKDB_TYPE_TIMESTAMP_MS: "TIMESTAMP_MS",
	C.DUCKDB_TYPE_TIMESTAMP_NS: "TIMESTAMP_NS",
	C.DUCKDB_TYPE_ENUM:         "ENUM",
	C.DUCKDB_TYPE_MAP:          "MAP",
	C.DUCKDB_TYPE_UNION:        "UNION",
	C.DUCKDB_TYPE_BIT:          "BIT",
	C.DUCKDB_TYPE_TIME_TZ:      "TIME_TZ",
	C.DUCKDB_TYPE_TIMESTAMP_TZ: "TIMESTAMP_TZ",
}

esevastyanov avatar Mar 05 '24 15:03 esevastyanov

We are not going to go ahead with this until DuckDB 0.10.1 is out since it will have some critical fixes for us. It's currently scheduled for next week.

begelundmuller avatar Mar 12 '24 09:03 begelundmuller