duckdb-wasm and arrow does weird things with some numeric constants

[broneill]

What happens?

When running a query with a literal containing a decimal point, the number is parsed as if the decimal point wasn't there at all.

To Reproduce

run: duckdb.table('airports.csv') -> {
  select: num is 123.4
  limit: 1
}

Produces this result:

num
1,234

As a workaround, 1234e-1 can be used instead.

OS:

Windows

Malloy Client:

Web-based VSCode

Malloy Client Version:

???

Database Connection:

DuckDB

[mtoy-googly-moogly]

@broneill I can't reproduce this running the vscode extension, is it possible you are running a very very old version of the extension?

run: duckdb.sql("select 1")->{ select: n is 123.4 }

Produces ...

[
  {
    "n": 123.4
  }
]

[broneill]

I installed the extension a few hours before I found this issue. How can I see what version it is?

[mtoy-googly-moogly]

The only way I know is to look in the upper right of the Malloy pane in the install window ...

[broneill]

It looks like I'm running v0.2.1712075074 as well. I'm using Firefox, in case that would make a difference.

[mtoy-googly-moogly]

@broneill I appreciate the bug report, and there once was a bug like this. That bug was fixed eight months ago.

I cannot reproduce what you are seeing from the information in your bug report. The code you posted looks like non-working code to me, since duckdb.table('airports.csv') doesn't read a CSV file, but rather looks for a table called airports.csv in the current schema inside duckdb. I'm not sure what is going on, can we try this ...

Post a screen shot, press Show SQL on the run: in the vscode window where you are seeing this problem maybe we can figure out why I can't reproduce this locally

Or catch me on Slack some time and we'll work through it

[broneill]

I created a new decimal-test.malloy file, ran the query, but there's no SQL output, just HTML. I'm running Malloy using the first link provided in the README. Is there another way I should be running it?

[mtoy-googly-moogly]

I am not well versed in these web-only experiences work. I'll redirect this bug towards someone who is, this is definitely a problem with how you are getting your Malloy.

Find us on slack: https://malloy-community.slack.com/ and we'll figure it out.

[mtoy-googly-moogly]

@whscullin I don't know what is going on here, I cannot reproduce on my machine, but it is somehow related to the try-malloy setup

[mtoy-googly-moogly]

@broneill I have reproduced this now. It's something weird about the try version of Malloy. We are working on it.

If you have vscode installed, and download the malloy extension, this doesn't happen.,

[whscullin]

There's a bug somewhere either in Apache Arrow or DuckDB wasm. We're getting a DecimalBigNum back, and depending on how you ask it for its value, you can get either '1234' or 425289321781646713966358060312437982418n. Weirdly if you use 123.4 / 1, it appears to return correctly as a normal number value. Working on a repro case with the upstream libraries.

[whscullin]

My guess at what's happening is that duckdb-wasm is using the C++ version of Arrow, using the Arrow IPC format to transfer values from the WASM worker to JavaScript, and there is a disconnect somewhere in either the encoding in C++ or decoding in Javascript.

[whscullin]

I've opened https://github.com/duckdb/duckdb-wasm/issues/1703 to track this in DuckDb WASM.