TopicNet icon indicating copy to clipboard operation
TopicNet copied to clipboard

Fix multiprocessing in tests

Open Alvant opened this issue 5 years ago • 1 comments

Currently multiprocessing may lead to the fact that tests fail from time to time. It means that builds may be red even there is all OK with the code. The case is not yet clear, but there are several hypotheses about what can cause this instability:

  • multiprocessing + pytest
  • too many experiments with the same name are conducted on too little text collection

Эта проблема, по-видимому, возникает на стыке работы мультипроцессинга и пайтеста. Т.е. маловероятно, что кто-то, кроме нас, её вообще видит. Другой возможный вариант — тренировка большого числа экспериментов с одинаковым именем на очень маленькой коллекции, что тоже не является рядовым случаем использования библиотеки.

E. Egorov

Alvant avatar Feb 11 '20 20:02 Alvant

UPD: We do rerun on the failed tests and they pass. It seems that that issue is not entirely in our code but in how it (multiprocess library) interacts with other instances of itself inside bigartm and pytest.

Evgeny-Egorov-Projects avatar Jul 07 '20 08:07 Evgeny-Egorov-Projects