Alex Movila

and another issue: File "c:\projects\rasa\deeppavlov-master\deeppavlov\dataset_readers\odqa_reader.py", line 130, in _build_db Path(save_path).unlink() File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\pathlib.py", line 1287, in unlink self._accessor.unlink(self) PermissionError: [WinError 32] The process cannot access the file because it is being...

and another issue in odqa_reader.py line 142 I had to add IF NOT EXISTS: sql_table = "CREATE TABLE IF NOT EXISTS documents (id PRIMARY KEY, text);"

related: https://github.com/renatoviolin/xlnet

Any progress with this? In meantime we have even better models: RoBERTa and ERNIE 2.0

There are some other tips here to make BERT better suited for production: https://hanxiao.github.io/2019/01/02/Serving-Google-BERT-in-Production-using-Tensorflow-and-ZeroMQ/

I think the idea of quantization + pruning could be useful to make BERT smaller. Also for production there is need of concurrent requests and loading the model only at...

Related: Here is another paper for XML model which is available at huggingface Large Memory Layers with Product Keys (https://arxiv.org/abs/1907.05242) ""outperforms a baseline transformer model with 24 layers, while being...

Well we must consider production where we have 1000 users doing inference concurrently. Also not everyone has GPU.

Now we have Albert: ALBERT: A Lite BERT For Self-Supervised Learning of Language Representations

Yeah I modified a bit the code to expose the probability. A configurable threshold could help.