openreview-expertise
openreview-expertise copied to clipboard
Expertise API/Image: Cache or create a store of embeddings
We previously cached embeddings in an addon module for Redis but ran into some issues with it failing to fetch.
Now that we are executing the model serverless, we need a way for an expertise container to pull embeddings from, most likely Google Cloud, into its local filesystem.
A model-agnostic implementation may be difficult since the embedding step happens at the model level. This might have to happen in the create_dataset
step and only embed publications in the filesystem, and merge the 2 embedding files.