curl_cffi icon indicating copy to clipboard operation
curl_cffi copied to clipboard

Add support for Firefox

Open JodanJodan opened this issue 1 year ago • 13 comments

The latest curl_impersonate supports 6 Firefox versions, could these be added?

JodanJodan avatar May 13 '23 23:05 JodanJodan

Hi, this would also interest me. is there any way we can modify code to do this ourself?

codeman5000 avatar Sep 12 '23 09:09 codeman5000

Take a look at the makefile, it's fairly simple. Basically, it's just downloading the compiled .so files and then using cibuildwheel to build binary wheels.

perkfly avatar Sep 13 '23 13:09 perkfly

Hello,

thx for your email. I'll take a look at it. Your project helps me a lot in bypassing bot protection when doing requests.

greetz

codeman5000 avatar Sep 13 '23 19:09 codeman5000

Hello,

i think it is to complicated for me to get it working. i have no idea where these come from:

.preprocessed: curl_cffi/include/curl/curl.h curl_cffi/cacert.pem .so_downloaded touch .preprocessed

any idea if there will be more browser imitations available in the future?

kind regards

codeman5000 avatar Sep 14 '23 10:09 codeman5000

+1

I also think Firefox's TLS Fingerprint should also be an option, it's useful to impersonate Firefox in case we use curl_cffi as proxy to surf the web with Firefox too, can't just impersonate Chrome because it's easy to detect.

gunir avatar Oct 06 '23 00:10 gunir

Hello, I'm trying to add firefox. How can I get browser infos like ciphers, and curves?

lxntagram avatar Nov 14 '23 14:11 lxntagram

Actually you don’t need to worry about that, it’s all there in the upstream curl-impersonate.

The real problem is that Firefox uses NSS, while other browsers use boringssl, and curl can only be linked to one TLS lib at the same time. So there are two separate libcurl so files for each browsers.

For curl-cffi, it’s also difficult to link to these two different so files. What you need to do is to find a way to add the firefox in addition to the chrome one, while allowing they can be selected by the user at runtime, or at least before importing curl-cffi.

来自手机回复

dIscOrd @.***>于2023年11月14日 周二22:56写道:

Hello, I'm trying to add firefox. How can I get browser infos like ciphers, and curves?

— Reply to this email directly, view it on GitHub https://github.com/yifeikong/curl_cffi/issues/59#issuecomment-1810384367, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH4ZXZPBTVUPMS5BND2SZ3YEOBCNAVCNFSM6AAAAAAYAY6Y7SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJQGM4DIMZWG4 . You are receiving this because you were assigned.Message ID: @.***>

perkfly avatar Nov 14 '23 15:11 perkfly

Would it be easier to have a sibling package, e.g. curl_cffi_ff, which provides the Firefox version of curl-impersonate? I would think that avoids the duplicate symbols issue

bjia56 avatar Dec 17 '23 03:12 bjia56

Yes, it’s one possible way, I have tried it actually, but it’s not ideal, neither.

I was thinking that is it possible to use boringssl to emulate NSS? Then it would be perfect to add firefox.

来自手机回复

Brett Jia @.***>于2023年12月17日 周日11:11写道:

Would it be easier to have a sibling package, e.g. curl_cffi_ff, which provides the Firefox version of curl-impersonate? I would think that avoids the duplicate symbols issue

— Reply to this email directly, view it on GitHub https://github.com/yifeikong/curl_cffi/issues/59#issuecomment-1859023933, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH4ZX7AKWOZ5SASOWJN6P3YJZPGRAVCNFSM6AAAAAAYAY6Y7SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJZGAZDGOJTGM . You are receiving this because you were assigned.Message ID: @.***>

perkfly avatar Dec 17 '23 03:12 perkfly

I was thinking that is it possible to use boringssl to emulate NSS? Then it

I think it's possible, as BoringSSL is very very close to NSS, unlike OpenSSL it's impossible because OpenSSL doesn't allow to hide some ciphers and a lot of extensions.

gunir avatar Dec 17 '23 05:12 gunir

It turns out that ffi is happy build multiple .so wrappers within the curl_cffi package, so it's possible to build curl_cffi._wrapper_chrome and curl_cffi._wrapper_ff at the same time. image Much of the library will need to be tweaked slightly to generalize the assumption that there's only one wrapper. Additionally, Firefox does not support the certificate store: image

bjia56 avatar Dec 26 '23 00:12 bjia56

Cool, do you mind submitting a PR?

perkfly avatar Dec 26 '23 03:12 perkfly

Yeah I'll probably need a bit of time to clean things up but I'll open a PR soon

bjia56 avatar Dec 26 '23 04:12 bjia56