extension: make more resilient against constant page updates

[karlicoss]

Some sites (e.g. twitter.com) emit onUpdated all the time when scrolled, which causes constant requests to the backend and redrawing the sidebar. Might be nice to debounce it or something.

[karlicoss]

Ugh. Some pages might even triggle multiple onUpdated events with complete=true, e.g. https://github.com/karlicoss/promnesia/projects/1

[ankostis]

Yes, debounce seems preferable e.g. launch a timer for ~1.2sec and if not any new onUpdate event gets detected till then, only then fetch visits, otherwise, renew timer. Does that make sense?

[karlicoss]

Kind of -- I was thinking more of "fetch the first time, but don't fetch the next times". Otherwise you'd always get a 1 second lag after loading page.

Basically I think querying visits is okay (it's fast anyway), the annoying bit is when it causes redrawing the sidebar and promnesia UI -- it can be both slow but also frustrating. So perhaps would be easier to just cancel the query.

Another thing to think about is how to cover this with tests -- if we have more complicated logic, we really don't want to break it later. There are some end2end tests (some are even running on CI), but they are mostly manual in the sense that they require a graphical browser to run. Should be possible to add a test with annoying sites that refresh all the time. Or even better would be nice to add an 'artificial' html page that cause it, e.g. every 500 ms or something. I can also help with showing how to run end2end tests -- would be great to get someone else to know it :)

[karlicoss]

Zulip (zulipchat.com) has brought promnesia to a crawl during a bout of fast typing.. seems that it sends onUpdate on every character

[yoyurec]

Looks like https://github.com/karlicoss/promnesia/pull/307 fixes it. For ex. https://github.com/karlicoss/promnesia/projects/1:

6 requests VS 1

VS

[karlicoss]

Hmm, interesting that I can't reproduce the issue on twitter and Zulip anymore -- it seems that they emit perhaps more onUpdated than we wish, but not on every scroll/character typed Either way, seems that switching to webNavigation.onComplete makes it strictly better anyway, just was hoping for something to write a regression test against -- so let me know if you have a decent example :)

[karlicoss]

I think that shouldn't be an issue anymore, but feel free to reopen if it still happens