Support redirection to the destination, without tracking services as middleman

[Chinaski1]

@imTHAI commented on Tue Nov 30 2021

Hello,

I would like Adguard support the URL redirect with cleaning the tracking. Like ClearURLs add-on does. Indeed both Adguard (with $removeparam I believe ?) and ClearURLs can clean tracking code after the URL For example:

https://example.com?utm_source=newsletter1&utm_medium=email&utm_campaign=sale After the address is cleaned, it will look like this: https://www.amazon.com/dp/exampleProduct

But clearURLs also cleans in case of redirection. That's the purpose of my request: For example: If I click on a link like this one: https://dealabs.digidip.net/visit?url=https%3A%2F%2Fwww.amazon.fr%2Fdp%2FB098K16RX5%3Fsmid%3DA1X6FK5RDHNB96&ppref=https%3A%2F%2Fwww.dealabs.com%2Fbons-plans%2Fasus-zenbook-um425qa-14-fhd-amd-ryzen-7-5800h-ram-16go-512-go-w10-azerty-2256311

With ClearURLs it will be cleaned to: https://www.amazon.fr/dp/B098K16RX5?smid=A1X6FK5RDHNB96

Another example: http://track.effiliation.com/servlet/effi.redir?id_compteur=12305754&effi_id=0k003mdl0mwv&url=https%3A%2F%2Ffr.shopping.rakuten.com%2Foffer%3Faction%3Ddesc%26aid%3D9027491774%26productid%3D6956351445

With ClearURLs it will be cleaned to: https://fr.shopping.rakuten.com/offer?action=desc&aid=9027491774&productid=6956351445

(with Safari/Adguard, it will give me an error page since track.effiliation.com is blocked by my AdguardHome dns server)

Some remarks:

• Here is the filter list of ClearURLs that I wish we could adapt into Adguard
• ClearURLS developers explained here that, other than the $99 for a Apple dev account, they wouldn't be able to port their extension to Safari for now, because of the lack of webRequestBlocking API. I imagine AdGuard could do that because it acts "outside" of Safari, doesn't it?
• Someone on Telegram says it would be possible to do the magic in AdGuard with something #%#AG_onload!function(){window.location.href but I means nothing to me :p
• I found something similar to my request here in the Adguard/Windows GitHub. But I'm not sure it's exactly the same.

Thank you, Best regards,

---

@Chinaski1 commented on Tue Nov 30 2021

Hello there!

As you correctly pointed out, you can use the $removeparam rule to remove tracking parameters. We have also implemented a URL Tracking filter that can be used for this purpose.

---

@imTHAI commented on Tue Nov 30 2021

Hello there!

As you correctly pointed out, you can use the $removeparam rule to remove tracking parameters. We have also implemented a URL Tracking filter that can be used for this purpose.

Hello, I'm not sure I understand you answer. You agree $removeparam cannot achieve my purpose right ?

---

@ameshkov commented on Tue Nov 30 2021

Ah, I got it.

Well, we were thinking about some sort of a $rewrite parameter, but the problem with it is that it can be rather dangerous. We might consider it for trusted filters, though.

@Chinaski1 probably should be moved to the CoreLibs repo.

[imTHAI]

Hello,

The uBlockOrigin developer said he will consider adding this capability after Adguard: https://github.com/uBlockOrigin/uBlock-issues/issues/1784

But it appears it does it now, but not automatically: https://github.com/gorhill/uBlock/wiki/Strict-blocking#ability-to-parse-the-strict-blocked-url

[ameshkov]

Duplicates #1364, please upvote it if you need it.

[sfionov]

Since $urltransform is only for same-origin redirects, I'm reopening this issue.

[imTHAI]

I will not lie, I am disappointed :/ I’ve been waiting for this feature for 3 years, as I already said in #1364 asking for news. I do not understand how it was considered as duplicate when I even gave examples. But maybe my request wasn’t clear enough, because of my English. Sorry.

All this said, I have full confidence in the Adguard team and you, developers, are doing a fantastic job. I am a big fan of Adguard products and I will wait patiently for a new solution to be developed. I just hope not to have to wait again 3 years :( Thanks to Sergey to have reopened this request.

As I said previously, uBlock Origin can do it now, if I'm not mistaken. I’m just saying this in case it might give you a lead on what-how.

Thanks again.

[cxplay]

Any progress on this? I've noticed that CoreLibs v1.16.19 seems to address this issue, but how exactly does it work?

[imTHAI]

I would test under windows, updating in AdGuard v7.19. But I can’t find a doc explaining how it’s implemented.

[imTHAI]

@cxplay did you find out how to do it ?

[cxplay]

@cxplay did you find out how to do it ?

No yet

[AdamWr]

Here is information about changing origin - https://adguard.com/kb/general/ad-filtering/create-own-filters/#urltransform-modifier:~:text=this%3A%20%5C%2C.-,Changing%20the%20origin,-COMPATIBILITY

As stated above, normally $urltransform rules are only allowed to change the path and query parts of the URL. However, if the rule's regexp begins with the string ^http, then the full URL is searched and can be modified by the rule. Such a rule will not be applied if the URL transformation can not be achieved via an HTTP redirect (for example, if the request's method is POST).

For example, to redirect all searches from google.com to duckduckgo.com something like this:

||google.*/search?q=$urltransform=/^https?:\/\/.*google\..*\/search\?q=([^&]+)&?.*/https:\/\/duckduckgo.com\/?q=\$1/

can be used. It should redirects from https://www.google.com/search?q=adguard to https://duckduckgo.com/?q=adguard&ia=web

The problem might be in the case of encoded URL. Like in:

https://track.effiliation.com/servlet/effi.redir?id_compteur=12305754&effi_id=1646343493&url=https%3A%2F%2Ffr.shopping.rakuten.com%2Foffer%2Fshop%2F11769144290%2Fdyson-v8-absolute-aspirateur.html%3FsellerLogin%3DBoulanger
https://www.dealabs.com/visit/search/2854617

The &url= parameter has https%3A%2F%2Ffr.shopping.rakuten.com%2Foffer%2Fshop%2F11769144290%2Fdyson-v8-absolute-aspirateur.html%3FsellerLogin%3DBoulanger which is encoded and it looks like that without decoding it will be not correctly redirected, so it's necessary to replace encoded parts (perhaps there is a better way to do it, but I don't know):

/^https?:\/\/(?:[a-z0-9-]+\.)*?(?:track\.effiliation\.com\/servlet\/effi\.redir|dealabs\.digidip\.net\/visit\?url=)/$urltransform=/%3A/:/
/^https?:\/\/(?:[a-z0-9-]+\.)*?(?:track\.effiliation\.com\/servlet\/effi\.redir|dealabs\.digidip\.net\/visit\?url=)/$urltransform=/%2F/\//
/^https?:\/\/(?:[a-z0-9-]+\.)*?(?:track\.effiliation\.com\/servlet\/effi\.redir|dealabs\.digidip\.net\/visit\?url=)/$urltransform=/%3F/?/
/^https?:\/\/(?:[a-z0-9-]+\.)*?(?:track\.effiliation\.com\/servlet\/effi\.redir|dealabs\.digidip\.net\/visit\?url=)/$urltransform=/%3D/=/
/^https?:\/\/(?:[a-z0-9-]+\.)*?(?:track\.effiliation\.com\/servlet\/effi\.redir|dealabs\.digidip\.net\/visit\?url=)/$urltransform=/^https?:\/\/(?:[a-z0-9-]+\.)*?(?:effiliation\.com|dealabs\.digidip\.net).*url=([^&]*)/\$1/

I guess that it would be useful to have an option to decode URL. Also having some examples about redirecting to another origin in docs might be nice.

[cxplay]

I've noticed the urltransform. But, yes, there is no better solution for URL encoding.

[imTHAI]

@cxplay @sfionov says on Telegram that, indeed, the current version does not support urltransform of whole URL. Next version will support, if match part starts with ^http @AdamWr thanks to you for the issue opened. Thanks again to the devs for their amazing work.

[imTHAI]

Hello @ngorskikh Actually without #1915 fixed, this request is not really completed. Or you have to make 5 rules to get anything done as explained by @AdamWr Is it possible that you increase the priority of the #1915 ? Thks