tf-keras
tf-keras copied to clipboard
keras-team
How to create a dynamic embedding layer in keras ?
Describe the feature and the current behavior/state.
I'm a recommendation system engineer and I know there are many Keras Preprocessing Layer added since tf 2.3.0 .
Now I can use tf.keras.layers.StringLookup + tf.keras.layers.Embedding to convert a string to an embedding tensor .But it fix the range of string and the range of embedding , which can't add new string and its embedding automatically when training new data . It leads that I can't use keras to do online daily training because the training data contains new user_id feature and item_id feature .
So I want to know wether keras can support it ?
Will this change the current api? How?
Who will benefit from this feature?
- Do you want to contribute a PR? (yes/no): If someone can tell me how to do it , I will do it .
Summary, new entry cannot be added to StringLookup, which is needed for incremental training.
I'm not sure a truly dynamic pairing of vocabulary indices and learned embedding weights (especially in a smoothly differentiable way) would be a good fit for either StringLookup or the Embedding layer. Did you have an example of something you had in mind?
We do however want to support moving a learned embedding from one vocabulary indexing to another. This should be doable, though the code might be a bit tricky, you could pass the new vocabulary through the old lookup layer, and then use gather/scatter to update the embedding weights. We've had some discussion on whether we would like to introduce some utility for this, though I don't think we have a clear design.
I find a keras layer in tfra repo : https://github.com/tensorflow/recommenders-addons/blob/master/docs/api_docs/tfra/dynamic_embedding/keras/layers.md . So it's possible to use tf.keras.layers.StringLookup and this layer to make it ?
