worker based implementation for user_streak , detoxify and feature extraction

[melvinebenezer]

trafficstars

user_streak , detoxify and feature extraction make the main process very slow. Use redis-queue based worker implementation for these tasks

[hemangjoshi37a]

Hello @melvinebenezer,

Thank you for bringing this issue to our attention. I understand that the current implementation for user_streak, detoxify and feature extraction is causing the main process to become slow. I suggest implementing a redis-queue based worker to handle these tasks. This will help to offload some of the workload from the main process and improve the performance of the application.

Additionally, I recommend conducting some performance tests on the new worker implementation to ensure that it is performing as expected. It would be also good to measure the performance of the system before and after the change to see the improvement.

I would also like to suggest looking into other potential areas of optimization in the codebase that may be contributing to the slow performance. It could be beneficial to identify and address these issues in order to further improve the overall performance of the application.

Thanks.

[andreaskoepf]

link #906

[olliestanley]

Is this done now?

[melvinebenezer]

@olliestanley yes