mwdb-core
mwdb-core copied to clipboard
Make result counting optional or parallel
Feature Category
- [ ] Correctness
- [ ] User Interface / User Experience
- [x] Performance
- [ ] Other (please explain)
Describe the problem
After launching query, MWDB first requests for count of results to validate the query and show the count.
Counting usually takes much longer than fetching first page of results. It highly affects querying performance and sometimes prevents using more complicated queries because counting takes too long and reaches the timeout.
Describe the solution you'd like
- Fetch first page and run counting in parallel.
- If count query doesn't finish and user tries to run another query: cancel the count request before running next query.
Describe alternatives you've considered
I'm not sure how dropped connections during request processing are handled in Flask/uWSGI. If request is not interrupted, SQL query may still be running reducing the overall performance. In that case, we should consider making "count button/link" instead and don't run counting automatically.
YES! Love it. This will significantly improve the current situation
Maybe add new button quick search in search bar that makes query without making count of objects? Now we are consider to add drop down to the search bar with additional option for searching without counting but adding this option for only single button is for me unnecessary now. In future it is worth consider that when we start to thing about another possibilities of querying (like test query etc.)