similarity icon indicating copy to clipboard operation
similarity copied to clipboard

Published 20 hours ago • verified publisher icon tensorflow

Storage Refactor

Open owenvallis opened this issue 2 years ago • 1 comments

Design and implement refactored storage and indexer modules to ensure that:

  • Storage is scalable and portable, e.g., we can store and query in javascript, etc.
  • The data storage is independent of the indexer
  • Loading the data from storage should be able to apply the original TF dtypes to the embeddings.

Example solutions may include supporting a storage backend like Parquet files or using a lib like DuckDB.

owenvallis avatar May 26 '22 00:05 owenvallis

I would like to work on this issue. Can I get a outline about refactored storage to start solving this issue? @owenvallis

Srikeshram avatar Jul 05 '22 16:07 Srikeshram