Extension: search takes a long time to complete when there are many results

[telotortium]

It would be good to support some sort of pagination in the search. For example, if I happen to search for "neural" or something like that, it takes a long time to complete (since I have many entries in my database, currently containing around 15k visits, that pertain to neural networks). It would be good to show only the first 50 results or so, and only show the rest if you either request a new page or scroll down to the bottom (only if you want to support infinite scroll, which I don't think is that useful here).

Worse, the page locks up until everything is rendered, so you can't even refine your search easily without hard refreshing the page.

I know pagination is hard, but this would make the extension easier to use.

[karlicoss]

Yep agree. I've been delaying this mainly to avoid reinventing the wheel (especially the UI), but maybe a trivial implementation wouldn't take too much effort. I think backend is fast enough even if the pagination implemented in the simplest way possible (i.e. just request X next visits from a certain timestamp, without keeping db connection open etc). Especially if we add an index on datetime field

[telotortium]

• [ ] Frontend call to backend: promnesia/api.js at 346b2e08e04604adffb17ae51244cd1b1ec9015f · karlicoss/promnesia - need to add some sort of page query parameter.
• [ ] Backend - need to add query parameter to visited (promnesia/server.py at f1c44a29c06a9c36d7dddd5be54c244516fe3ee8 · karlicoss/promnesia) and modify the SQL query.
  • Probably best to use seek pagination with request fields limit and after_rowid to specify getting the first limit results that are ordered after the rowid passed to after_rowid (or the first limit results in general if after_rowid isn't passed).