CoreLibs icon indicating copy to clipboard operation
CoreLibs copied to clipboard

Remove complicated heuristics for $domain modifier

Open Yuki2718 opened this issue 1 year ago • 13 comments

I mean

$domain modifier matching target domain:

In some cases the $domain modifier can match not only the referrer domain, but also the target domain. This happens when all the following conditions are met:

  1. The request has the document content type
  2. The rule pattern does not match any particular domains
  3. The rule pattern does not contain regular expressions
  4. The $domain modifier contains only excluded domains, e.g. $domain=~example.org|~example.com

This seems to be the culprit of false positive I fixed.

STR:

  1. Add ://www2.etc-$document,domain=com|net|icu
  2. Set languages for Google search to Japanese (and maybe use Japanese VPN).
  3. Search for ETC利用照会サービス
  4. Click ログイン

login

blocked

IMO the heuristics is no more needed, as we now have $to and it should be decided by filter author in actual context.

Yuki2718 avatar Apr 14 '24 11:04 Yuki2718

As I recall there were lots of $cookie rules that were using $domain to point to a target domain?

ameshkov avatar Apr 14 '24 13:04 ameshkov

As I recall there were lots of $cookie rules that were using $domain to point to a target domain?

They can be converted to $to.

Yuki2718 avatar Apr 14 '24 13:04 Yuki2718

But are they converted in other filter lists? Like uBO lists for instance?

ameshkov avatar Apr 14 '24 13:04 ameshkov

uBO doesn't support $cookie in the first place.

Yuki2718 avatar Apr 14 '24 13:04 Yuki2718

Sorry, I may be confusing things now, but I recall that in uBO filters something similar was used for other rule types, maybe it was $removeparam?

ameshkov avatar Apr 14 '24 14:04 ameshkov

I don't remember, but I can say we uBO team always assume $domain works as $from, meaning it is used only for base domain than target.

Yuki2718 avatar Apr 14 '24 14:04 Yuki2718

Got it, thank you! I generally like the idea and this whole heuristic was made for compatibility reasons back in the day.

ameshkov avatar Apr 14 '24 14:04 ameshkov

https://github.com/uBlockOrigin/uBOL-home/issues/156 Somewhat related and AGMV3 is also affected. Do all the AG products now support $to and can I safely change these $domain rules in Japanese filter for malicious sites to $to? E.g. /\/t_([0-9A-Za-z]{32})\?token=\1/$document,domain=cn|com|net|top.

Yuki2718 avatar Jul 29 '24 15:07 Yuki2718

All save for Safari-based: https://github.com/AdguardTeam/SafariConverterLib/issues/60 (but I think it can't be properly implemented there anyway)

ameshkov avatar Jul 30 '24 17:07 ameshkov

It's a bit problematic if we have to use $domain for Safari and $to for MV3. Fortunately there are not too many $document rules with $domain so maybe we can use hint or !#if.

Yuki2718 avatar Aug 01 '24 09:08 Yuki2718

Maybe you're talking about a problem with the referer:

https://github.com/AdguardTeam/CoreLibs/issues/1579 https://github.com/AdguardTeam/CoreLibs/issues/1421

In AdGuard 4.2+ add-on, opening links from Google causes a strict block prompt again for good sites. I haven't tested yet replace domain/from with to= to unlock opening good pages and prompt warning for scam/malware.

krystian3w avatar Aug 01 '24 09:08 krystian3w

Safari most likely won't be able to apply $to properly anyway

ameshkov avatar Aug 01 '24 10:08 ameshkov

Related problem with 3p filter https://github.com/AdguardTeam/AdguardFilters/issues/185543#issuecomment-2271599288

Alex-302 avatar Aug 09 '24 17:08 Alex-302