rasa
rasa copied to clipboard
RasaHQ
Persist ConveRT model in a local persistent cache
Description of Problem: Currently, the ConveRTFeaturizer downloads the ConveRT model from a remote repository and stores it in the temporary cache directory of tensorflow hub which is located at $HOME/.cache. Since, this is a temporary cache directory it gets wiped out on every system restart. This is even more problematic for docker setups as you can imagine much more frequent docker container restarts during experimentation because of which the ConveRT model needs to be downloaded everytime from scratch. There has also been an issue in the past where the ConveRT model was taken offline from the central remote repository and it caused some assistants in production to break.
Overview of the Solution: There are a couple of solutions possible:
- We add a configuration option to
ConveRTFeaturizercalledcache_dir(or something similar) which when specified, stores the ConveRT model to a persistent directory. This directory path should be set as the value forTFHUB_CACHE_DIRenvironment variable so that during training the model gets persisted at this location automatically and during inference the model is loaded from this path. Once this option is added, docs should suggest users to use this option necessarily in production. - We could use a default persistent cache directory if Rasa uses one and persist / load the model from there just like we do in option 1.
Definition of Done:
- [ ] Tests are added
- [ ] Feature described the docs
- [ ] Feature mentioned in the changelog
Exalate commented:
pksvv commented:
Can this be done since polyai has already discontinued the convert models.
https://github.com/PolyAI-LDN/polyai-models
After much consideration, the PolyAI https://www.polyai.com/ team has decided to take down the ConveRT models from the public domain.
On Thu, Mar 18, 2021 at 1:58 PM Daksh Varshneya @ .***> wrote:
Description of Problem : Currently, the ConveRTFeaturizer downloads the ConveRT model from a remote repository and stores it in the temporary cache directory of tensorflow hub which is located at $HOME/.cache. Since, this is a temporary cache directory it gets wiped out on every system restart. This is even more problematic for docker setups as you can imagine much more frequent docker container restarts during experimentation because of which the ConveRT model needs to be downloaded again from scratch. There has also been an issue in the past where the ConveRT model was taken offline from the central remote repository and it caused some assistants in production to break.
Overview of the Solution : There are a couple of solutions possible:
We add a configuration option to ConveRTFeaturizer called cache_dir (or something similar) which when specified, stores the ConveRT model to a persistent directory. This directory path should be set as the value for TFHUB_CACHE_DIR environment variable so that during training the model gets persisted at this location automatically and during inference the model is loaded from this path. Once this option is added, docs should suggest users to use this option necessarily in production.
We could use a default persistent cache directory if Rasa uses one and persist / load the model from there just like we do in option 1.
Definition of Done :
- Tests are added
- Feature described the docs
- Feature mentioned in the changelog
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/RasaHQ/rasa/issues/8229, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACEQ3C5ZWWUNQNO5VFIQSDTTEG2RTANCNFSM4ZMETMFA .
– Thanks Vipul Gaur
Exalate commented:
dakshvar22 commented:
@pksvv You can still use a community hosted model like this. Just pass the link as value for model_url option of ConveRTFeaturizer.
Exalate commented:
pksvv commented:
ok we've implemented similar solution for the same.
On Thu, Mar 18, 2021 at 3:27 PM Daksh Varshneya @ .***> wrote:
@pksvv https://github.com/pksvv You can still use a community hosted model like this https://github.com/davidalami/ConveRT/releases/download/1.0/nocontext_tf_model.tar.gz. Just pass the link as value for model_url option of ConveRTFeaturizer.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/RasaHQ/rasa/issues/8229#issuecomment-801786864, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACEQ3C3YGLIOKHGNDP3BUL3TEHE77ANCNFSM4ZMETMFA .
– Thanks Vipul Gaur
