[discussion] Reduce listeners, increase number of query params checked on every url

[apiraino]

This is a open discussion on how we can plan for a better scalability and perfomance of the extension.

Looking at the code, I think the way we manage query parameters removal does not scale well. Brief recap:

• We have a number of listeners attached to onBeforeRequest: each listeners filters some query params to a specific URL regexp. Example for "*://*.instagram.com/*" filter out the query param igshid.

• We have some more listeners that apply to any URL the browser is processing. Example: remove the query params utm_* from any URL.

Instead of adding a new listener everytime a user asks us to support a new website and remove one or two parameters, I was wondering if it would make more sense to reduce the number of listeners and increase the number of query params checked on any URL. Example: blindly check and remove all these query params if_id, utm_*, efg, eid, fref,... on every URL the browser is processing.

The reason I'm asking this is manyfold:

1. Would reducing the number of listeners decrease the footprint of the addon on the browser? I don't know the internals of a browser but I guess that the url filters we give to the listeners is nothing but a fast regexp itself.
2. Expected advantage: if we reduce the listeners theoretically to one ("all urls") we can then extend more easily the number of query params checked, simply adding an item to a list.
3. How can we profile the extension performances? How can I measure if it's more lightweight adding a long chain of query params check to one listener, rather than attaching many listeners, each checking just a couple of query params?

and ...

4. I'm thinking that if we centralize and reduce the points where the regexp checks happen, we can prepare for a future rewrite of the "core" with something more performant (form example using a statically compiled language)

5. reducing the amount of code logic would enable us to more easily port the extension to other browsers (something that I still didn't investigate) that support the WebExt API

[apiraino]

@rugk thank you so much for your thoughts and analysis, great insights and suggestions :+1: Now I know a bit better where to focus for my experiments.

Namely I really like this:

refactor/create a wrapper or similar around browser.webRequest.onBeforeRequest.addListener in general, that makes it easier for you to add rulesets.

As a positive side effect, this could allow me to refactor the link cleaning when triggered by the user selecting "Copy and clean link" from the context menu: https://github.com/apiraino/link_cleaner/blob/5a60fc9dba2459fae80aa9c7a372b3b7c3bb8077/copy.js#L64-L89

(As a reminder for myself) I don't like too much how I currently manage that and I'd like to improve it somehow.