$strict-first-party $urltransform and $removeparam issue on macOS 15

[billkewl]

Please answer the following questions for yourself before submitting an issue

• [X] Filters were updated before reproducing an issue
• [X] I checked the knowledge base and found no answer
• [X] I checked to make sure that this issue has not already been filed

AdGuard version

Version 2.15.1.1731 release

Browser version

Firefox 130, Safari 18.0

OS version

macOS 15

Ad Blocking

No response

Privacy

No response

Social

No response

Annoyances

No response

Security

No response

Other

No response

Language-specific

No response

Which DNS server do you use?

DNS protection disabled

DNS protocol

None

Custom DNS

No response

What Stealth Mode options do you have enabled?

Block trackers, Remove tracking parameters, Hide your search queries, Send Do-Not-Track signals, Self-destruction of third-party cookies, Self-destruction of first-party cookies, Disable cache for third-party requests, Block the third-party Authorization header, Block WebRTC, Block Push API, Block Location API, Block Java, Hide your Referrer from third-parties, Hide your User Agent, Mask your IP address, Remove X-client-Data header from HTTP request, Protect from DPI

Support ticket ID

No response

Issue Details

Steps to reproduce: $strict-first-party + $urltransform issue

1. add user rule ||itemimg-dcs.adss-sys.com/itemimg^$urltransform=/\-120\.jpg/.jpg/,strict-first-party
2. open https://itemimg-dcs.adss-sys.com/itemimg/DC020/A0DC00008MXJ/03_080-120.jpg for example
3. refresh page. nothing happned.
4. Replace $strict-first-party with $~3p or $to=adss-sys.com url still not transformed.
5. remove $strict-first-party. url finally transformed

---

$removeparam issue

1. add nhk.jp#$# *{pointer-events: unset !important;} to unblock right click and nhk.jp/static/assets/images^$removeparam to remove resize parameters
2. right click open any image on NHK webpage. Here I use https://www.nhk.jp/p/omusubi/ts/NJ1W7VQ6W9/blog/bl/pm97YxdDv1/bp/pK9rpJQ6A0/ as an example
3. https://www.nhk.jp/static/assets/images/newblogposting/ts/NJ1W7VQ6W9/NJ1W7VQ6W9-editor_94f0226250f583a17078908b35b1eb7e.png?width=1080&height=1350 opened but parameters are not removed before a refresh on the image page

Expected Behavior

$strict-first-party + $urltransform issue url should be able to transformed with $strict-first-party modifier. similar uBO rule ||itemimg-dcs.adss-sys.com/itemimg^$uritransform=/\-120\.jpg/.jpg/,strict1p is working on uBO

---

$removeparam issue

nhk.jp#$# *{pointer-events: unset !important;}
nhk.jp/static/assets/images^$removeparam

is compatible with uBO and is working as expected. AdGuard is expected to be the same.

Turning on/off hide referer from 3rd parties stealth mode option did not affect the results.

Actual Behavior

$strict-first-party + $urltransform dont work together at all. I haven't confirmed if $strict-first-party is working with other modifier yet.

$removeparam only work after a refresh. This seems to be a rare case as $removeparam can strip parameters from most urls without issue

Screenshots

Screenshot 1

Additional Information

No response

[billkewl]

Similar problem? happens on google with href-sanitizer scriplet

www.google.*#%#//scriptlet('href-sanitizer', '#main a[href^="/url?q=http"]', '?q')

Only after refresh can i get clean link of search results. Not always happen but it does sometimes. No issue with the same rule on uBO.

[AlexandrPkhm]

@billkewl Hello!

Could you please try to check if the issue still persists on the latest Nightly version of AdGuard for Mac which is available via the following link: https://agrd.io/mac_nightly.

If the issue still persists, please provide the application logs. We need them in order to diagnose and troubleshoot this issue. Here's what we need you to do:

1. Click AdGuard icon in the menu bar → Gear → Advanced → Logging → Logging level → Debug.
2. Reproduce the problem, then remember the exact time it happened.
3. Menu → Advanced → Logging → Export Logs and System Info.….
4. Send this file to [email protected]:
   • include [mac] keyword and ISSUE_NUMBER in the subject of your email
   • specify the exact time when the issue occurred

[billkewl]

I dont want to use nightly as it may very likely break something after downgrading to stable release and I did export logs before for myself. It contains too much private info which I'm hesitant to share. Can you try to reproduce this in your local environment?

---

Well. I briefly tested the nightly version on my friends macOS :(

Application version: 2.16.0.1814 nightly (CL-1.16.29, DNS-2.5.46)
    Scriptlets 1.11.27
    ContentScript 2.0.6
    ExtendedCss 2.0.52
    StealthScript 1.2.0
    UserScriptWrapper 1.2.24
    AdBlockedPage 1.2.12

$removepara issue only happens on the desktop App. Browser extension doesnt have the problem. $urltransform is not available for browser extension so there's no way to test.

Here's what I found so far

---

$strict-first-party $urltransform issue

• ||itemimg-dcs.adss-sys.com/itemimg^$urltransform=/\-120\.jpg/.jpg/,to=itemimg-dcs.adss-sys.com is working when opening in a new page but at the same time, all thumbnails in https://ap-story.jp/ap/item/i/A0DC00008MMA were redirected to their large images.
• $~third-party works only if right click on an image to open (open thumbnails on https://ap-story.jp/ap/item/i/A0DC00008MMA for example) probably it's due to empty referer of new page. What's weird is that when I checked the 302 request from right click open, the header was Referer https://ap-story.jp/ not itemimg-dcs.adss-sys.com
• $strict-first-party doesn't work at all, right click open or open in new page, due to empty referer again? Unlike uBO, AdGuard only has strict first party modifier. Overall. $to $strict1p and $~3p don't have inconsistent referer origin matching rule and can't handle empty referer header requests well.

---

$removeparam issue I have no idea since the rule is working with browser extension and other $removeparam user rules seem working fine. On desktop App, refresh the opened image page (with parameters) or open the image url can get clean url successfully as well. Maybe first time opened image is loaded from cache and AdGuard doesn't apply user rule fast enough?

---

href-sanitizer rule. Do have time to test since it's not happening all the time and I dont have time to try over and over on friends Mac.

[AlexandrPkhm]

@billkewl Hello!

Please accept our apologies for the delayed response. We have done some testing and would like to clarify the following:

$strict-first-party + $urltransform issue

It is not a bug, as $strict-first-party modifier is not supported by majority of our application and is not implemented in our CoreLibs applications yet. You can confirm the following in our Knowledge base Basic modifiers table.

$removeparam issue

Seems to be a bug. We will pass this info to our development team.

href-sanitizer scriplet issue

Could you please clarify what you would like to achieve with the following rule www.google.*#%#//scriptlet('href-sanitizer', '#main a[href^="/url?q=http"]', '?q')? If you would like to strip everything from the link after q parameter, it is currently unsupported in our implementation of href-sanitizer. If you would like to achieve a different result, please provide a more detailed steps to reproduce for this issue.

[billkewl]

Could you please clarify what you would like to achieve with the following rule www.google.*#%#//scriptlet('href-sanitizer', '#main a[href^="/url?q=http"]', '?q')? If you would like to strip everything from the link after q parameter, it is currently unsupported in our implementation of href-sanitizer.

This rule is from ublock origin's privacy filter I believe it's used on google to get rid of google's redirections and use the q's value as href's value. I have no problem with uBO. So it's confirmed to be not supported yet? Do I need to open another ticket for this? I have no idea how to describe this though.

[AlexandrPkhm]

@billkewl

So it's confirmed to be not supported yet? Do I need to open another ticket for this?

As far as we understand your use case, yes. You can create a task in the AdGuard scriptlets repo, you can call it something like: Add support for the ?q query parameter in the href-sanitizer scriptlet.

As to $removeparam issue. It looks like a Corelibs bug, since it's reproduced on MacOS and Windows apps. So we migt have to transfer your task to the Corelibs repo.

[AlexandrPkhm]

@billkewl

Our Corelibs dev team confirmed your suspicion regarding $removeparam issue. Opening an image via Right click and Open Image in New Tab opens the image from the cache and puts the URL of where it was originally loaded in the address bar. So there is nothing we can can do using rules to avoid that.

[billkewl]

@billkewl

Our Corelibs dev team confirmed your suspicion regarding $removeparam issue. Opening an image via Right click and Open Image in New Tab opens the image from the cache and puts the URL of where it was originally loaded in the address bar. So there is nothing we can can do using rules to avoid that.

But the same rule has no issue removing parameters on uBO. Is there really nothing can be done about it? Most sites aren't like this and parameters are stripped as intended.

[AdamWr]

If I'm not wrong, in uBO, $removeparam is applied to every type of request by default, but in AdGuard only to document type. You can try to use rule like this:

nhk.jp/static/assets/images^$image,document,removeparam

Seems to works fine on my end with AdGuard for Windows.

[billkewl]

@billkewl

As far as we understand your use case, yes. You can create a task in the AdGuard scriptlets repo, you can call it something like: Add support for the ?q query parameter in the href-sanitizer scriptlet.

support for query parameter is implemented according to scriptlet wiki

Syntax

example.org#%#//scriptlet('href-sanitizer', selector[, attribute])

    selector — required, a CSS selector to match the elements to be sanitized, which should be anchor elements (<a>) with href attribute.
    attribute — optional, default to text:
        text — use the text content of the matched element,
        [attribute-name] copy the value from attribute attribute-name on the same element,
        ?parameter copy the value from URL parameter parameter of the same element's href attribute.

However, it's not working consistently. The Google search page rule doesn't work at times. I don't know how to reproduce it at 100% success rate yet.

[billkewl]

If I'm not wrong, in uBO, $removeparam is applied to every type of request by default, but in AdGuard only to document type. You can try to use rule like this:

nhk.jp/static/assets/images^$image,document,removeparam

Thank you @AdamWr I can confirm this is working on Firefox with Adguard for Mac as well.

I found this in Adguard documents $removeparam rules without [content type modifiers](https://adguard.com/kb/general/ad-filtering/create-own-filters/#content-type-modifiers) will only match requests where the content type is document.

Do you know what else modifiers work like this?

[AdamWr]

Do you know what else modifiers work like this?

Probably only $permissions and $removeparam modifiers.

[billkewl]

||pbs.twimg.com^$doc,image,urltransform=/name=(medium|large)/name=orig/ is not working while it has no problem in uBO $urltransform has similar issue maybe?

Probably only $permissions and $removeparam modifiers.

[AdamWr]

I need steps to reproduce. I have tested it here - https://x.com/visegrad24/status/1858844015191351572, with rule like this:

||pbs.twimg.com^$urltransform=/name=(small|medium|large|900x900)/name=orig/

and

||pbs.twimg.com^$urltransform=/name=(small|medium|large|900x900)/name=orig/

In both cases, when I open image in a new tab, I see that it's redirected from https://pbs.twimg.com/media/Gcvwx31WMAAGU4b?format=jpg&name=small to https://pbs.twimg.com/media/Gcvwx31WMAAGU4b?format=jpg&name=orig

Maybe it's already in cache, please try to clear cache, or check in private/incognito mode.

[billkewl]

@AdamWr I tried on AdGuard for Mac Version 2.16.0.1845 nightly and the rule worked for a while. Then I tried to add ~3p modifier, it stopped working, and it didn't started to work when I undo it. Maybe a nightly bug?

[AdamWr]

Then I tried to add ~3p modifier, it stopped working, and it didn't started to work when I undo it.

I see that the rule with ~3p doesn't work if I open image (https://pbs.twimg.com/media/Gcvwx31WMAAGU4b?format=jpg&name=small) directly in the address bar (it works if I open image using context menu on x.com), but when I remove ~3p then it starts working again.

---

By the way, it's not important, but now I noticed that in my previous answer I have added the same rule two times, but one of them should be:

||pbs.twimg.com^$document,image,urltransform=/name=(small|medium|large|900x900)/name=orig/

[AdamWr]

Regarding issue with ~3p modifier, it looks like a bug, I have opened issue here - https://github.com/AdguardTeam/CoreLibs/issues/1931

[billkewl]

@AdamWr Do you know if it's a redirect rule bug or Firefox bug that when I right click on a redirected (orig) url to save image, it saves the orig image while drag the image to local file explorer, the small size image is saved. Not 100% of time but at times, drag and drop also saves orig image

[AdamWr]

I don't know.

[billkewl]

Can you reproduce on your end? Drag and drop the twitter orig image. Is it the in orig or smaller size? I hate when potential bugs happen not 100% the time :(

[AdamWr]

I need detailed steps to reproduce it.

[billkewl]

Add rule ||pbs.twimg.com/media^$doc,image,urltransform=/name=(small|medium|large|\d+x\d+)/name=orig/,~3p

Open a small size image such as https://pbs.twimg.com/media/GUUF1bDW4AA0Pdj?format=jpg&name=small on Firefox v132

Drag the http 302 redirected image from browser to file explorer

Right click on the image to save the same image

Compare image quality

Repeat few times until both results (saved images are the same and not the same)

[AdamWr]

Thank you.

||pbs.twimg.com/media^$doc,image,urltransform=/name=(small|medium|large|\d+x\d+)/name=orig/,~3p

I have to use a rule without ~3p, otherwise it's not redirected. In Firefox, if I save a file then there is large file (better quality), if I drag and drop to desktop then it's smaller file. In Chrome in both cases there is larger file.

[billkewl]

In Firefox, if I save a file then there is large file (better quality), if I drag and drop to desktop then it's smaller file. In Chrome in both cases there is larger file.

This is what happened her, most of the time. However sometimes they are the same. I have no idea how to trigger the other outcome

I have to use a rule without ~3p, otherwise it's not redirected.

I can use with ~3p on Adguard for Mac. But https://github.com/AdguardTeam/AdguardForMac/issues/1488#issuecomment-2486989894 this happens at times. I have to restart Adguard, wait for some time until the rule takes effect