openreview-expertise icon indicating copy to clipboard operation
openreview-expertise copied to clipboard

Expertise API/Image: Cache or create a store of embeddings

Open haroldrubio opened this issue 1 month ago • 0 comments

We previously cached embeddings in an addon module for Redis but ran into some issues with it failing to fetch.

Now that we are executing the model serverless, we need a way for an expertise container to pull embeddings from, most likely Google Cloud, into its local filesystem.

A model-agnostic implementation may be difficult since the embedding step happens at the model level. This might have to happen in the create_dataset step and only embed publications in the filesystem, and merge the 2 embedding files.

haroldrubio avatar Jan 09 '25 19:01 haroldrubio