datastation

datastation copied to clipboard

Creating the indexes after the insert is substantially faster on the taxi dataset. 50s overall vs 130s (when creating the index before inserting)

eatonphil

eatonphil

If the user doesn't fill in an s3 output bucket we should try to create an s3 bucket for them. If that fails then the user has to create their...

eatonphil

Can't upgrade React to 18 until this.

eatonphil

File panel should support glob matches or something like it

1

What happens when you have parquet files split up or logs that are split among a bunch of different files?

eatonphil

Javascript / Module support

3

This works with bash helper (code at end of this note): ```js import {readFile} from "node:fs/promises" const content = await readFile('test.txt',{encoding:'utf8'}) ``` but when I add the functions to import/export...

aheissenberger

Add a language Markdown

1

This will allow to add Documentation and in a second step to access and display values from other panels

aheissenberger

The visual transform panel has a bunch of limits (no HAVING, no multiple GROUPs). While it would be great to support these all eventually, in the meantime (and probably forever)...

eatonphil

Figure out a baseline on github actions with hyperfine (see https://github.com/cube2222/octosql/blob/main/benchmarks/benchmarks.sh) and start testing these kinds of things: https://github.com/multiprocessio/datastation/pull/164.

eatonphil

There are missing integration tests for: * [ ] Connecting to databases over SSH * [ ] Reading SQLite files over SSH (database panel) * [x] Prometheus * [x] Snowflake

eatonphil