Error [RuntimeError]: Aborted(). Build with -sASSERTIONS for more info

[Mause]

I've been playing around with trying to get duckdb-wasm running in netlify functions, but I'm having some issues with the wasm side

I'm currently using @duckdb/duckdb-wasm version "1.16.0", the code is in this repo: https://github.com/Mause/duckdb_website_search

Jul 25, 01:10:06 PM: 585b1635 ERROR  Error [RuntimeError]: Aborted(). Build with -sASSERTIONS for more info.
    at Me (/var/task/node_modules/@duckdb/duckdb-wasm/dist/duckdb-node-mvp.worker.cjs:2:75567)
    at A_ (/var/task/node_modules/@duckdb/duckdb-wasm/dist/duckdb-node-mvp.worker.cjs:2:84990)
    at tp (/var/task/node_modules/@duckdb/duckdb-wasm/dist/duckdb-node-mvp.worker.cjs:3:26451)
    at wasm://wasm/0493fb0a:wasm-function[348]:0x13f89
    at wasm://wasm/0493fb0a:wasm-function[13772]:0xcfa79a
    at wasm://wasm/0493fb0a:wasm-function[13798]:0xcfedf9
    at jp (/var/task/node_modules/@duckdb/duckdb-wasm/dist/duckdb-node-mvp.worker.cjs:3:28285)
    at wasm://wasm/0493fb0a:wasm-function[13939]:0xd0f4a5
    at lp (/var/task/node_modules/@duckdb/duckdb-wasm/dist/duckdb-node-mvp.worker.cjs:3:27175)
    at wasm://wasm/0493fb0a:wasm-function[13751]:0xcf8873

[ankoh]

That error is not really helpful, for sure. I haven't looked at netlify functions yet but I'd start with building duckdb-wasm with debug symbols to get meaningful function names in the stack trace. We don't have that wasm module prebuilt unfortunately.

[Mause]

@ankoh there doesn't seem to be much in the way of build instructions anywhere, should i be doing make wasm_debug js_debug or somesuch?

[Mause]

Okay, progress!

Jul 26, 01:04:50 PM: 24790052 ERROR  Error [RuntimeError]: Aborted(native code called abort())
    at T (/var/task/api/duckdb-wasm/duckdb-node-mvp.worker.cjs:2:118142)
    at zl (/var/task/api/duckdb-wasm/duckdb-node-mvp.worker.cjs:3:9424)
    at kh (/var/task/api/duckdb-wasm/duckdb-node-mvp.worker.cjs:4:19638)
    at std::__terminate(void (*)()) (wasm://wasm/0c1fe5de:wasm-function[332]:0x35ed6)
    at std::terminate() (wasm://wasm/0c1fe5de:wasm-function[330]:0x35eb4)
    at duckdb::web::io::WebFileSystem::WebFileHandle::~WebFileHandle() (wasm://wasm/0c1fe5de:wasm-function[50270]:0x19574cc)
    at duckdb::web::io::WebFileSystem::WebFileHandle::~WebFileHandle().1 (wasm://wasm/0c1fe5de:wasm-function[50383]:0x1964ed8)
    at std::__2::default_delete<duckdb::web::io::WebFileSystem::WebFileHandle>::operator()(duckdb::web::io::WebFileSystem::WebFileHandle*) const (wasm://wasm/0c1fe5de:wasm-function[56873]:0x1b65018)
    at std::__2::unique_ptr<duckdb::web::io::WebFileSystem::WebFileHandle, std::__2::default_delete<duckdb::web::io::WebFileSystem::WebFileHandle> >::reset(duckdb::web::io::WebFileSystem::WebFileHandle*) (wasm://wasm/0c1fe5de:wasm-function[54793]:0x1b14f41)
    at std::__2::unique_ptr<duckdb::web::io::WebFileSystem::WebFileHandle, std::__2::default_delete<duckdb::web::io::WebFileSystem::WebFileHandle> >::~unique_ptr() (wasm://wasm/0c1fe5de:wasm-function[54778]:0x1b1446b)

Unfortunately doesn't seem that much more helpful?

[ankoh]

Ah wait, lets start simple here: Does executing a Select 42 return a correct result in netlify when opening the database in-memory without a path? WebFileHandle points to an interaction with the web filesystem which immediately adds plenty of other possible failure points.

[Mause]

Yep, that seems to work fine - are there docs describing how I'm supposed to be opening on disk databases with duckdb-wasm?

[ankoh]

I haven't really written many docs for DuckDB-Wasm. :) But we can check something else first: Can you please scan the following file in the netlify worker: https://shell.duckdb.org/data/tpch/0_01/parquet/lineitem.parquet

Just download the file, upload it the same way you uploaded the database file and then try 2 things: First reference the plain file in the SQL text and if that doesn't work, register the file handle manually using: https://shell.duckdb.org/docs/classes/index.AsyncDuckDB.html#registerFileHandle

[ankoh]

If that works, we're quite close.

[Mause]

@ankoh what exactly should I be passing for the handle? The path to the file? (The parameter is very weakly typed)

[Mause]

Okay, using registerFileBuffer I'm able to read the parquet file fine (see https://duckdb-website-search.netlify.app/.netlify/functions/short and https://github.com/Mause/duckdb_website_search/blob/main/api/short.ts)

Trying to do the same with the database gives me a read-only filesystem error (see https://duckdb-website-search.netlify.app/.netlify/functions/index?q=Hello%20world and https://github.com/Mause/duckdb_website_search/blob/main/api/index.ts).

Not particularly surprising, but I'm not sure which path I'm supposed to take from here

[ankoh]

DuckDB-Wasm will just run fs.readSync on the file handle. https://github.com/duckdb/duckdb-wasm/blob/15c8075b5f82d3d32386c5c1cd425b5473524de6/packages/duckdb-wasm/src/bindings/runtime_node.ts#L155

In node.js, you can therefore pass the file descriptor returned by fs.open to duckdb-wasm. It's terribly typed, because this argument is either the fd on node.js or the html 5 file handle returned from a file picker in the browser.

[Mause]

Okay, in the end I was able to make this work using that method, but only after copying it to the /tmp filesystem, to work around the readonly filesystem error

[ankoh]

Great! Then we have to try this again after https://github.com/duckdb/duckdb-wasm/issues/908