adfilt icon indicating copy to clipboard operation
adfilt copied to clipboard

Parameters that cannot be removed from the Bilibili website

Open DeepChirp opened this issue 1 year ago • 43 comments

Describe the problem below this line as meticulously and detailed as possible (incl. pagelinks if any)

https://www.bilibili.com/video/BV1g24y1h7zc If the user logs in and opens any video page on Bilibili, then a parameter called vd_source will be added. Although I added the rule ||bilibili.com^$removeparam=vd_source in Actually Legitimate URL Shortener Tool, this parameter still cannot be removed. I'm guessing that some script on the site is at work, so someone who knows about scripting needs to be able to deal with this.

Add screenshots below if needed

No response

Add a screenshot of the extension's logger

No response

Which adblocker(s) did you use when testing this?

uBlock Origin

Adblocker version(s)

No response

Which filterlists did you use? Failing to tell this will temporarily close the report until it has been told.

Actually Legitimate URL Shortener Tool

Which browser(s) did you use when testing this?

Microsoft Edge (Manifest V2)

Browser version(s)

No response

Which OS(s) did you use when testing this?

No response

OS version(s)

No response

DeepChirp avatar Jan 19 '23 09:01 DeepChirp

I can't sign up as it appears to require a phone number. My hypothesis is that it uses history.replaceState to change the displayed URL without loading the new page (and thus bypassing any browser extensions). If you copy and paste the URL into a new tab, is the parameter removed? That is all speculation, and I probably am wrong.

iam-py-test avatar Jan 19 '23 21:01 iam-py-test

I can't sign up as it appears to require a phone number. My hypothesis is that it uses history.replaceState to change the displayed URL without loading the new page (and thus bypassing any browser extensions). If you copy and paste the URL into a new tab, is the parameter removed? That is all speculation, and I probably am wrong.

possible. Parameters were removed first, then reappeared. For this situation, can we prevent some code in the script from running? When it is convenient for you, I can provide you with the account number.

DeepChirp avatar Jan 20 '23 03:01 DeepChirp

Sorry for the delay. If you feel comfortable with that, you can email it to me at iam-py-test[@]protonmail[.]com If not, I can try to investigate without an account.

iam-py-test avatar Jan 21 '23 16:01 iam-py-test

Sorry for the delay. If you feel comfortable with that, you can email it to me at iam-py-test[@]protonmail[.]com If not, I can try to investigate without an account.

There may be a time difference between the two of us. It is best when you and I are online at the same time, you send the QR code here, and then I scan it to log in. If you log in directly with account number and password, you also need to verify my mobile phone.

DeepChirp avatar Jan 22 '23 01:01 DeepChirp

Sorry for the delay. I'm in Eastern Timezone, if that matters. I should be available from now to 11:30 (my timezone). Just tell me when you are ready and I can try to login. If that doesn't work, can you open the Logger, reproduce the issue, and post back with the log? I'm sorry, but working across timezones is hard

iam-py-test avatar Jan 22 '23 14:01 iam-py-test

Sorry for the delay. I'm in Eastern Timezone, if that matters. I should be available from now to 11:30 (my timezone). Just tell me when you are ready and I can try to login. If that doesn't work, can you open the Logger, reproduce the issue, and post back with the log? I'm sorry, but working across timezones is hard

You can send me the login QR code as soon as you get online today so I'm still online.

DeepChirp avatar Jan 23 '23 11:01 DeepChirp

@shenzhiming88 here is the code (I think I did it right) Code:

It will expire fairly soon though

iam-py-test avatar Jan 23 '23 11:01 iam-py-test

@shenzhiming88 here is the code (I think I did it right) Code:

It will expire fairly soon though

Oh sorry, I just saw that. Please send it again.

DeepChirp avatar Jan 23 '23 12:01 DeepChirp

Sorry for the delay

iam-py-test avatar Jan 23 '23 12:01 iam-py-test

Sorry for the delay

Very strange, I scan this QR code and there is no response

DeepChirp avatar Jan 23 '23 12:01 DeepChirp

It timed out.

iam-py-test avatar Jan 23 '23 12:01 iam-py-test

Looks like it worked. Thanks

iam-py-test avatar Jan 23 '23 12:01 iam-py-test

I see it. Investigating...

iam-py-test avatar Jan 23 '23 12:01 iam-py-test

I see it. Investigating...

OK, I guess some script causes this.

DeepChirp avatar Jan 23 '23 12:01 DeepChirp

Off-topic, but does ||bilibili.com^$removeparam=spm_id_from seem to break anything? It seems to cause "video not found errors" on my end Right now, I need to get ready for work, but I will look into this later. Sorry

iam-py-test avatar Jan 23 '23 12:01 iam-py-test

Off-topic, but does ||bilibili.com^$removeparam=spm_id_from seem to break anything? It seems to cause "video not found errors" on my end Right now, I need to get ready for work, but I will look into this later. Sorry

This parameter doesn't seem to break anything, at least on my computer.

DeepChirp avatar Jan 23 '23 12:01 DeepChirp

Odd. Maybe it has something to do with me using Firefox and you using Edge. Not relevant to this issue, just noticed that.

iam-py-test avatar Jan 23 '23 12:01 iam-py-test

Odd. Maybe it has something to do with me using Firefox and you using Edge. Not relevant to this issue, just noticed that.

However, another script that removes the extra parameters of the web page also includes this parameter, and it seems that no problem report has been seen. Also, please come back to investigate this issue when you have time.

DeepChirp avatar Jan 23 '23 12:01 DeepChirp

I think I have isolated the script responsible, and that completely removing replaceState is the best solution. Please try this rule and report back:

bilibili.com##+js(set,history.replaceState,noopFunc)

It's not a great rule, but I don't think it breaks anything

iam-py-test avatar Jan 23 '23 23:01 iam-py-test

I think I have isolated the script responsible, and that completely removing replaceState is the best solution. Please try this rule and report back:

bilibili.com##+js(set,history.replaceState,noopFunc)

It's not a great rule, but I don't think it breaks anything

This rule can indeed remove vd_source, but since uBlock cannot remove the parameters of the video link opened from the recommendation bar on the right, I used a script using the replaceState method. This rule will cause the script to fail. I don't think there is a good way to handle this.

DeepChirp avatar Jan 24 '23 02:01 DeepChirp

Ok, thanks. I will see if there is a better solution. I have been busy this week, but will try to figure this out soon

iam-py-test avatar Jan 24 '23 11:01 iam-py-test

Can you (Mostly shenzhiming) post an instance of what a URL with vd_source in it looks like? For example https://www.bilibili.com/video/BV1g24y1h7zc/?vd_source=johnmadden?

DandelionSprout avatar Mar 04 '23 23:03 DandelionSprout

Can you (Mostly shenzhiming) post an instance of what a URL with vd_source in it looks like? For example https://www.bilibili.com/video/BV1g24y1h7zc/?vd_source=johnmadden?

This parameter is related to the user's personal information, so I found a example on the Internet: https://www.bilibili.com/video/BV1WZ4y1v7pq/?spm_id_from=333.337.search-card.all.click&vd_source=a0a2bf40b0c968029d22c1cceee21c53 (from https://nga.178.com/read.php?tid=34162479&rand=263)

DeepChirp avatar Mar 05 '23 00:03 DeepChirp

Hmm… seems like uBlock Origin successfully removes vd_source from that link when I'm not logged in. So I have limited ideas on how to fix this.

Does the problem remain if you use these entries in User Filters (For almost every non-inline script I could find on the page that didn't break something)?

*/log/$domain=bilibili.com
||cm.bilibili.com^$all
/bili-collect.js$all
||bilibili.com^*/data/collector^$all
||boss.hdslb.com^$all
||bilibili.com/x/click-interface^$all
||bilibili.com/*/is_forbid^$all
||bilibili.com/*/log-reporter.js$all
*/laputa-header/$all
||hdslb.com/*/pcdn-$all
||hdslb.com/*/nc-loader-$all

DandelionSprout avatar Mar 05 '23 01:03 DandelionSprout

Hmm… seems like uBlock Origin successfully removes vd_source from that link when I'm not logged in. So I have limited ideas on how to fix this.

Does the problem remain if you use these entries in User Filters (For almost every non-inline script I could find on the page that didn't break something)?

*/log/$domain=bilibili.com
||cm.bilibili.com^$all
/bili-collect.js$all
||bilibili.com^*/data/collector^$all
||boss.hdslb.com^$all
||bilibili.com/x/click-interface^$all
||bilibili.com/*/is_forbid^$all
||bilibili.com/*/log-reporter.js$all
*/laputa-header/$all
||hdslb.com/*/pcdn-$all
||hdslb.com/*/nc-loader-$all

Still exists. This seems difficult to deal with.

DeepChirp avatar Mar 05 '23 01:03 DeepChirp

This is caused by the vf function on line 197 of https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js. I haven't found a way to touch that with scriptlets, and blocking the entire script or overwriting history.replaceState breaks the page.

Trace replaceState blob:https://www.bilibili.com/744c15ef-0e23-4c0e-9f6a-7c04d75fdde2:5 (this line can be ignored)
ht https://s1.hdslb.com/bfs/static/jinkela/long/js/sentry/sentry-5.7.1.min.js:2
vf https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
mf https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
replace https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
transitionTo https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
confirmTransition https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
r https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
r https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
f https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
f https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
r https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
gf https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
confirmTransition https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
r https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
r https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
f https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
yf https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
f https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
r https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
r https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
gf https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
confirmTransition https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
transitionTo https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
replace https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
replace https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
e https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
p https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:190
O https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:190
x https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:190
n https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
s https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
(Async: promise callback)
n https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
s https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
exports https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
exports https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
checkLoginInfoForHeaderV3 https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
e https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
p https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:190
O https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:190
x https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:190
n https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
s https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
exports https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
exports https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
init https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
e https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
p https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:190
O https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:190
x https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:190
n https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
s https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
(Async: promise callback)
n https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
s https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
exports https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
exports https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
mounted https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
Bn https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:7
an https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:7
insert https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:7
S https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:7
ga https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:7
_update https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:7
r https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:7
get https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:7
t https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:7
mount https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:7
$mount https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:7
$mount https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:7
https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
onReady https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
onReady https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:197
n https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:180
n https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1
https://s1.hdslb.com/bfs/static/jinkela/video/stardust-video.366216d83af1f0d17e2505f09cab46978fafc9c1.js:1

iam-py-test avatar Mar 05 '23 13:03 iam-py-test

I tried https://www.bilibili.com/video//?spm_id_from= redirector=https://www.bilibili.com/video/$1?vd_source=0 but it do not help redirect even not remove spm_id_from

KamiyaMinoru avatar Mar 28 '23 12:03 KamiyaMinoru

https://www.bilibili.com/video//?spm_id_from= redirector=https://www.bilibili.com/video/$1?vd_source=0

In ClearURLs or uBlock Origin?

iam-py-test avatar Mar 28 '23 20:03 iam-py-test

I have made a (kind-of poorly made) extension which should fix this issue. It's barely been tested, and isn't great code, and I am not extremely happy asking people to install a separate extension just for this (maybe there's still a way to prevent this from happening, or something a custom uBo scriptlet can do?), so buyer beware. Also needs to be loaded as a dev mode extension in FF/Chrome https://github.com/iam-py-test/bilibili-url-clean

iam-py-test avatar Mar 28 '23 22:03 iam-py-test

https://www.bilibili.com/video//?spm_id_from= redirector=https://www.bilibili.com/video/$1?vd_source=0

In ClearURLs or uBlock Origin?

ubo

KamiyaMinoru avatar Mar 28 '23 23:03 KamiyaMinoru