inputs icon indicating copy to clipboard operation
inputs copied to clipboard

dates not detected in arrow tables

Open Fil opened this issue 1 year ago • 1 comments

A bug where dates are shown as numbers in the table; it should have been covered by #263, but in this case we have field.type.unit==0 instead of 1.

The bug can be seen in Framework with the following page:

---
sql:
  launches: data/launches.csv
---

# Rocket launches 🚀

```sql echo
SELECT * FROM launches;
```

Capture d’écran 2024-10-15 à 18 36 21

I can't find the documentation for what this "unit 0" means. The source code https://github.com/apache/arrow/blob/main/js/src/fb/time.ts seems to indicate that these are seconds. If I change https://github.com/observablehq/inputs/blob/7ae53c757080b99ca3b498653cf2bc49bf8125a1/src/table.js#L389 to accept 0 as a date, the dates appear correctly in the table.

Capture d’écran 2024-10-15 à 18 35 06

Fil avatar Oct 15 '24 16:10 Fil

Ended up here from a data source duckdb -> parquet -> observable framework Plots: it worked on 2024-03-31 (framework 1.5.1, duckdb python 0.10.1): both duckdb SQL DATE & TIMESTAMP were of type js Date in the browser document. Now only TIMESTAMP is: DATE is a number (framework 1.13, duckdb 1.1)/

(I failed to reproduce the old build: I lost my local _npm cache, and the implicit npm:apache-arrow import gets the latest version, and I failed to force another version: it loads both and only the latest is used, and explicit imports were not supported on framework 1.5, and with 1.6 and explicit import apache-arrow after npm install apache-arrow@15 still gets both versions loaded and latest used.. -> is there really a way to force an old apache-arrow?)

Reduced repro to:

```js
Inputs.table(await sql`SELECT current_date as thedate, current_localtimestamp() as thetimestamp`)
```

Image

-> it would be great to support the DateUnit 0: Day

Niluge-KiWi avatar Jan 29 '25 01:01 Niluge-KiWi