mwdb-core icon indicating copy to clipboard operation
mwdb-core copied to clipboard

Make result counting optional or parallel

Open psrok1 opened this issue 2 years ago • 1 comments

Feature Category

  • [ ] Correctness
  • [ ] User Interface / User Experience
  • [x] Performance
  • [ ] Other (please explain)

Describe the problem

After launching query, MWDB first requests for count of results to validate the query and show the count.

Counting usually takes much longer than fetching first page of results. It highly affects querying performance and sometimes prevents using more complicated queries because counting takes too long and reaches the timeout.

Describe the solution you'd like

  • Fetch first page and run counting in parallel.
  • If count query doesn't finish and user tries to run another query: cancel the count request before running next query.

Describe alternatives you've considered

I'm not sure how dropped connections during request processing are handled in Flask/uWSGI. If request is not interrupted, SQL query may still be running reducing the overall performance. In that case, we should consider making "count button/link" instead and don't run counting automatically.

psrok1 avatar Sep 07 '22 18:09 psrok1

YES! Love it. This will significantly improve the current situation

ITAYC0HEN avatar Sep 08 '22 12:09 ITAYC0HEN

Maybe add new button quick search in search bar that makes query without making count of objects? Now we are consider to add drop down to the search bar with additional option for searching without counting but adding this option for only single button is for me unnecessary now. In future it is worth consider that when we start to thing about another possibilities of querying (like test query etc.)

KWMORALE avatar Nov 18 '22 07:11 KWMORALE