Lance as an append-only time series database?

[mutexstream]

Is lance a good fit for building an append-only time series database, where batches of data are continuously appended as "versions" of the dataset? Are there other ways to achieve this?

Context: I want a file format that allows me to query for data in a given time range and also to append new data to the end of the current file as new data arrives.

[eddyxu]

Hey, @mutexstream . Currently each "append" operation create a new file and seal it after finishes.

Before the WAL feature is landed, it would expect each append operation insert a large chunk of data to be more efficient. Does it fit the workload you are expecting?

[changhiskhan]

Closing due to inactivity

[mutexstream]

Missed the comment. No inserting large chunk doesn't fit my use case, but I decided to move on to other tasks for now.