Pablo A. Delgado

Are there any updates towards this goal? It will be very useful to have one implementation of a storage backend so people can start contributing with different time series databases,...

I am encountering this issue too. I am following the recommendation for leaving the head node configured with resources: {"CPU": 0} to avoid scheduling additional tasks on it. ( https://docs.ray.io/en/master/cluster/guide.html#deployment-guide...

I am also interested in some form of integration between LanceDB + Mosaic Streaming.

+100 to this. You can get some ideas/inspiration/mechanics from this project https://docs.mosaicml.com/projects/streaming/en/stable/index.html It has great ideas on shuffling, data sharding and caching.