protected site

[udiudiudiudi]

hi, can you maybe add a feature that depends on the protection of the site I will be able to execute and it will include the flags needed to bypass that specific security

if its cloudflare or datadome for example

I tried to implement this fix for datadome https://github.com/berstend/puppeteer-extra/issues/182

and currently its not working

[GiveDaData]

Make your bot pass https://bot.incolumitas.com/ first (By comparing the result from actual browser and your bot) Then make sure the IP used is good

Depending on your frequency. You may also want to randomize the broswer fingerprint. But, I think fp stealth is not the focus of the devs at the moment.

One pointer I could give is look into "webgl.vendor" evasion and change the default "intel" to a valid GPU tag. (You can obtain from https://bot.incolumitas.com/ under "videoCard" property)

[udiudiudiudi]

Hi, I use residential ips, so i don't think that's the problem however I am keen to understand how do you do this "randomize the broswer fingerprint" what would you randomize, in order to not have a static fingerprint?

[GiveDaData]

They are listed in https://bot.incolumitas.com/

TCP/IP Fingerprint TLS Fingerprint Browser Fingerprint Canvas Fingerprint WebGL Fingerprint

because stealth plugin sets a static value for some of these fp so the bot can be detected if datadome look for the GPU property and found the same strings used in the plugin. because not many people has that gpu, and if you make enough requests to raise alert then you'd get blocked. Thus, you need to blend into the other users making requests.

and you'd have to try for your own custome solution, because not all sites with datadome have the same rules. they can customize what to block and what to let pass

[udiudiudiudi]

i recently implemented bspine, but didn't think about rotating the GPU, which I get correctly that is what you are saying.

additionally, I read about this solution which I implemented https://github.com/berstend/puppeteer-extra/issues/182

and it didn't solve, probably because what you say about the GPU rotation

[GiveDaData]

if your bot has the exact same result as a real broswer when visiting https://bot.incolumitas.com/ then rotating finger print is what you need next (also you need to make the finger print not unique, https://amiunique.org/)

not sure about what you mean by "bspine" in terms of bot detection evasion

[udiudiudiudi]

bspline = simulate human-like mouse

1. https://medium.com/analytics-vidhya/how-to-easily-bypass-recaptchav2-with-selenium-7f7a9a44fa9e
2. https://github.com/guilhermebferreira/selenium-notebooks/blob/master/Mouse%20move%20by%20b-spline%20interpolation.ipynb

[GiveDaData]

awsome, thank you for the links. I will learn up on it.

[udiudiudiudi]

just to finilize,

1. you claim intel gpu is not the best and I should rotate GPUs
2. check that my fingerprint keeps on changing on https://bot.incolumitas.com/

[udiudiudiudi]

i made a test like you said

zardaxt.py - Passive TCP/IP Fingerprint => changed ja3 - Passive SSL/TLS Fingerprint => not changed fingerprintjs - Browser Fingerprint => changed canvas fingerprint => not changed webgl fingerprint => not changed

for canvas, i understand that adblock would solve it? https://digiwonk.gadgethacks.com/how-to/canvas-fingerprinting-stop-webs-sneakiest-tracking-tool-your-browser-0156506/

for webgl, you told me to rotate graphic driver, correct? is there a list somewhere for common graphic drivers? github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-stealth/evasions/webgl.vendor

for ja3, I saw this bc-security.org/post/ja3-s-signatures-and-how-to-avoid-them/

as for https://amiunique.org/fp - even my own browser showed me unique - You are unique among the 3604317

anything else you would suggest me on doing?

[GiveDaData]

just to finilize,

1. you claim intel gpu is not the best and I should rotate GPUs
2. check that my fingerprint keeps on changing on https://bot.incolumitas.com/

1. I said the default one used by stealth plugin is not common. If you make a lot of requests with that property, you will get caught out by datadome. So, at least try using some common ones. If this is not enough, try rotating through a few common ones.
2. some values do not stay the same, you can find out more about each fp by clicking on the links next to the title.

[udiudiudiudi]

thank you, i also thought about rotating browsers, and rotating browser versions

i will update you on the results.

[GiveDaData]

i made a test like you said

zardaxt.py - Passive TCP/IP Fingerprint => changed ja3 - Passive SSL/TLS Fingerprint => not changed fingerprintjs - Browser Fingerprint => changed canvas fingerprint => not changed webgl fingerprint => not changed

for canvas, i understand that adblock would solve it? https://digiwonk.gadgethacks.com/how-to/canvas-fingerprinting-stop-webs-sneakiest-tracking-tool-your-browser-0156506/

for webgl, you told me to rotate graphic driver, correct? is there a list somewhere for common graphic drivers? github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-stealth/evasions/webgl.vendor

for ja3, I saw this bc-security.org/post/ja3-s-signatures-and-how-to-avoid-them/

as for https://amiunique.org/fp - even my own browser showed me unique - You are unique among the 3604317

anything else you would suggest me on doing?

I am still new to the finger print evasions. what worked for me was I made my bot's result from https://bot.incolumitas.com/ as close to real chrome result as possible. and this worked for a while until datadome's AI caught up and I then had to use real webgl.vendor property.

[udiudiudiudi]

so i thought of taking - List of Vendor / Renderer - https://gist.github.com/TimvanScherpenzeel/eb296c564ef592883472eec8291b078e

and rotate them, this is something that worked for you?

IF YOU HAVE ANOTHER LIST I WOULD APPRECIATE AS WELL

additionally, now datadome ai, doesn't catch you?

[udiudiudiudi]

what code did you use to spoof the webgl please?

secondly, here is a list of more WebGL - https://store.steampowered.com/hwsurvey/videocard/

[GiveDaData]

what code did you use to spoof the webgl please?

secondly, here is a list of more WebGL - https://store.steampowered.com/hwsurvey/videocard/

I used chrome to visit https://bot.incolumitas.com/ and simply copied the values.

when you are on that page, search for "videoCard"

or use this one: https://bot.sannysoft.com/ it lists WebGL Vendor and WebGL Renderer

Regarding the GPU lists. I am not sure how relevant it is to match the driver versions So I copied actual values instead of writing up my own.

[udiudiudiudi]

I found this list of GPUs - https://store.steampowered.com/hwsurvey/videocard/

[udiudiudiudi]

I am making progress on the Ja3 Fingerprinting randomization, can we communicate somewhere else directly? @GiveDaData

[taewookim]

By any chance, any update? Would love to offer help

Having this issue as well. Tried solution from SO.. no luck

[brianschardt]

Having an issue as well