odqa example gives RuntimeError in Win10

[alex-movila]

I run this code: from deeppavlov import configs from deeppavlov.core.common.file import read_json from deeppavlov import configs, train_model

model_config = read_json(configs.doc_retrieval.en_ranker_tfidf_wiki) model_config["dataset_reader"]["data_path"] = "SentenceCorpus/unlabeled_articles/plos_unlabeled" model_config["dataset_reader"]["dataset_format"] = "txt" model_config["dataset_reader"]["save_path"] = "odqa/enwiki.db" model_config["dataset_iterator"]["load_path"] = "odqa/enwiki.db" doc_retrieval = train_model(model_config)

I get this:

2019-08-07 15:21:53.184 INFO in 'deeppavlov.dataset_readers.odqa_reader'['odqa_reader'] at line 57: Reading files... 2019-08-07 15:21:53.185 INFO in 'deeppavlov.dataset_readers.odqa_reader'['odqa_reader'] at line 134: Building the database... 2019-08-07 15:21:54.48 INFO in 'deeppavlov.dataset_readers.odqa_reader'['odqa_reader'] at line 57: Reading files... 2019-08-07 15:21:54.50 INFO in 'deeppavlov.dataset_readers.odqa_reader'['odqa_reader'] at line 134: Building the database... Traceback (most recent call last): File "", line 1, in File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="mp_main") File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\runpy.py", line 85, in run_code exec(code, run_globals) File "C:\Projects\RASA\BERT_Chatbot\rnet_test\test.py", line 12, in doc_retrieval = train_model(model_config) File "c:\projects\rasa\deeppavlov-master\deeppavlov_init.py", line 31, in train_model train_evaluate_model_from_config(config, download=download, recursive=recursive) File "c:\projects\rasa\deeppavlov-master\deeppavlov\core\commands\train.py", line 92, in train_evaluate_model_from_config data = read_data_by_config(config) File "c:\projects\rasa\deeppavlov-master\deeppavlov\core\commands\train.py", line 58, in read_data_by_config return reader.read(data_path, **reader_config) File "c:\projects\rasa\deeppavlov-master\deeppavlov\dataset_readers\odqa_reader.py", line 81, in read self._build_db(save_path, dataset_format, expand_path(data_path)) File "c:\projects\rasa\deeppavlov-master\deeppavlov\dataset_readers\odqa_reader.py", line 146, in _build_db workers = Pool(num_workers) File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\multiprocessing\context.py", line 119, in Pool context=self.get_context()) File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\multiprocessing\pool.py", line 176, in init self._repopulate_pool() File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\multiprocessing\pool.py", line 241, in _repopulate_pool w.start() File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\multiprocessing\process.py", line 112, in start self._popen = self._Popen(self) File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\multiprocessing\popen_spawn_win32.py", line 46, in init prep_data = spawn.get_preparation_data(process_obj._name) File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\multiprocessing\spawn.py", line 143, in get_preparation_data _check_not_importing_main() File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main is not going to be frozen to produce an executable.''') RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

2019-08-07 15:21:54.429 INFO in 'deeppavlov.dataset_readers.odqa_reader'['odqa_reader'] at line 57: Reading files... 2019-08-07 15:21:54.430 INFO in 'deeppavlov.dataset_readers.odqa_reader'['odqa_reader'] at line 134: Building the database...

[alex-movila]

and another issue: File "c:\projects\rasa\deeppavlov-master\deeppavlov\dataset_readers\odqa_reader.py", line 130, in _build_db Path(save_path).unlink() File "C:\Installed\Anaconda3\envs\RASA_Chatbot\lib\pathlib.py", line 1287, in unlink self._accessor.unlink(self) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Projects\RASA\FAQ_BERT_Chatbot\rnet_test\odqa\enwiki.db' 2019-08-07 15:34:53.759 INFO in 'deeppavlov.dataset_readers.odqa_reader'['odqa_reader'] at line 57: Reading files...

[alex-movila]

and another issue in odqa_reader.py line 142 I had to add IF NOT EXISTS: sql_table = "CREATE TABLE IF NOT EXISTS documents (id PRIMARY KEY, text);"

[Kolpnick]

Hello, @alex-movila! Sorry for the late response. We tested this issue on DeepPavlov version 1.2.0 and didn’t encounter the described problem. If you are still having difficulties with this, please let us know.