ipfs-companion icon indicating copy to clipboard operation
ipfs-companion copied to clipboard

[deferred] Observe WebRequests and populate LFU Cache

Open whizzzkid opened this issue 2 years ago • 3 comments

Parent Issue: https://github.com/ipfs/ipfs-companion/issues/1152

ETA: 2023-05-12

### Tasks
- [x] Observe webRequests via the non-blocking API
- [ ] Implement LFU  maintain list of LFU URLs that can be serviced by companion.
- [ ] Validate LFU thresholds to have sufficient content that exceeds the blockable requests.

Discussion Notes:

  • [ ] Explore LRFU Cache
  • [ ] Explore 2Q Cache

whizzzkid avatar Feb 15 '23 09:02 whizzzkid

@whizzzkid : I'm wondering if we're over complicating this. A couple of thoughts:

  1. We can store 5k regexes right? Do we have a sense of whether 5k+ domains even have "IPFS support"? Related, over the course of someone's browsing session, how many unique domains (regardless if they have "IPFS support") does one have? My guess is that 5k is plenty big enough. Before we do work here, lets just get a metric on how big we find this to be in practice.
  2. Once the 5k limit is hit, what happens when you try to add another regex? Do you get an error? Do you have to choose what regex to evict so thee is space for the new regex you want to add?

BigLep avatar Jun 03 '23 05:06 BigLep

@BigLep

Do we have a sense of whether 5k+ domains even have "IPFS support"? Related, over the course of someone's browsing session, how many unique domains (regardless if they have "IPFS support") does one have? My guess is that 5k is plenty big enough.

I agree, it does seem enough, however the second part answers this question.

Once the 5k limit is hit, what happens when you try to add another regex?

It's an error, this limit might change in the future if we're to believe chrome team. The cache would help us determine which rules are good to remove which needs to still be in there. In fact I was looking at 2Q cache implementation with a TTL, i.e. auto-purging all rules to free up space for more.

whizzzkid avatar Jun 06 '23 03:06 whizzzkid

Ok, got it. It's good to know we get an error and thus have a metric we can report.

I don't think we should put energy on being fancy for now. I think we should ship and then can improve this later if we see the need in the metrics.

BigLep avatar Jun 06 '23 14:06 BigLep