curl-impersonate icon indicating copy to clipboard operation
curl-impersonate copied to clipboard

Published 20 hours ago • verified publisher icon lwthiker

Using with PHP

Open fusionconsulting opened this issue 2 years ago • 7 comments

I'm trying to use lib-impersonate-chrome with a PHP script but so far no luck.

Red Hat Enterprise Linux release 9.1 (Plow)

I've installed patchelf and run the following: /snap/patchelf/current/bin/patchelf --set-soname libcurl.so.4 /usr/local/lib/libcurl-impersonate-chrome.so.4

Which results in: /snap/patchelf/current/bin/patchelf: /snap/core/current/lib/x86_64-linux-gnu/libc.so.6: version 'GLIBC_2.33' not found (required by /usr/local/lib/libcurl-impersonate-chrome.so) /snap/patchelf/current/bin/patchelf: /snap/core/current/lib/x86_64-linux-gnu/libc.so.6: version 'GLIBC_2.34' not found (required by /usr/local/lib/libcurl-impersonate-chrome.so)

`ldd --version` outputs:

ldd (GNU libc) 2.34

So it seems that the correct GLIBC version is present.

Also tried:

LD_PRELOAD=/usr/local/lib/libcurl-impersonate-chrome.so CURL_IMPERSONATE=chrome101 php 'print_r(curl_version());'

This shows no errors when running a subsequent command so I believe it's loading the lib correctly. However, after both, the SSL version is showing:

[ssl_version] => OpenSSL/3.0.1

Any ideas what's wrong here?

fusionconsulting avatar Feb 16 '23 09:02 fusionconsulting

I'm also having an issue and don't have a clue why, as it was working previously.

No matter if I use LD_PRELOAD or not, I'm only getting default curl inside php:

✅ Raw curl (with LD_PRELOAD) works as expected (BoringSSL)

$ LD_PRELOAD=/usr/local/lib/libcurl-impersonate-chrome.so CURL_IMPERSONATE=chrome104 curl --version
curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/7.84.0 BoringSSL zlib/1.2.11 brotli/1.0.9 nghttp2/1.46.0
Release-Date: 2020-01-08
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli HTTP2 HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL UnixSockets
WARNING: curl and libcurl versions do not match. Functionality may be affected.

✅ Impersonate curl works as expected (BoringSSL)

$ curl_chrome110 --version
curl 7.84.0 (x86_64-pc-linux-gnu) libcurl/7.84.0 BoringSSL zlib/1.2.11 brotli/1.0.9 nghttp2/1.46.0
Release-Date: 2022-06-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL threadsafe UnixSockets

❌ But from PHP it doesn't (OpenSSL)

$ LD_PRELOAD=/usr/local/lib/libcurl-impersonate-chrome.so CURL_IMPERSONATE=chrome104 php -r 'print_r(curl_version());'
Array
(
    [version_number] => 476160
    [age] => 5
    [features] => 12568477
    [ssl_version_number] => 0
    [version] => 7.68.0
    [host] => x86_64-pc-linux-gnu
    [ssl_version] => OpenSSL/1.1.1f
    [libz_version] => 1.2.11
    [protocols] => Array
        (
            [0] => dict
            [1] => file
            [2] => ftp
            [3] => ftps
            [4] => gopher
            [5] => http
            [6] => https
            [7] => imap
            [8] => imaps
            [9] => ldap
            [10] => ldaps
            [11] => pop3
            [12] => pop3s
            [13] => rtmp
            [14] => rtsp
            [15] => scp
            [16] => sftp
            [17] => smb
            [18] => smbs
            [19] => smtp
            [20] => smtps
            [21] => telnet
            [22] => tftp
        )

    [ares] =>
    [ares_num] => 0
    [libidn] => 2.2.0
    [iconv_ver_num] => 0
    [libssh_version] => libssh/0.9.3/openssl/zlib
    [brotli_ver_num] => 16777223
    [brotli_version] => 1.0.7
)

jlcd avatar Feb 28 '23 19:02 jlcd

I managed to configure it here you can see.

LD_PRELOAD=/usr/local/lib/libcurl-impersonate-chrome.so CURL_IMPERSONATE=chrome101 php -r 'print_r(curl_version());'

Array ( [version_number] => 480256 [age] => 9 [features] => 1370063517 [ssl_version_number] => 0 [version] => 7.84.0 [host] => x86_64-pc-linux-gnu [ssl_version] => BoringSSL [libz_version] => 1.2.11 [protocols] => Array ( [0] => dict [1] => file [2] => ftp [3] => ftps [4] => gopher [5] => gophers [6] => http [7] => https [8] => imap [9] => imaps [10] => mqtt [11] => pop3 [12] => pop3s [13] => rtsp [14] => smb [15] => smbs [16] => smtp [17] => smtps [18] => telnet [19] => tftp )

[ares] => 
[ares_num] => 0
[libidn] => 
[iconv_ver_num] => 0
[libssh_version] => 
[brotli_ver_num] => 16777225
[brotli_version] => 1.0.9

)

but I can't find how to use it from php. when i run

dump(curl_version()); [ "version_number" => 479488 "age" => 9 "features" => 364890013 "ssl_version_number" => 0 "version" => "7.81.0" "host" => "x86_64-pc-linux-gnu" "ssl_version" => "OpenSSL/3.0.2" "libz_version" => "1.2.11" "protocols" => array:25 [▶] "ares" => "" "ares_num" => 0 "libidn" => "2.3.2" "iconv_ver_num" => 0 "libssh_version" => "libssh/0.9.6/openssl/zlib" "brotli_ver_num" => 16777225 "brotli_version" => "1.0.9" ]

Please someone help me solve it, I can't find how to load it so that php can take it

vhpm18 avatar Mar 31 '23 04:03 vhpm18

Having the same issue: https://github.com/lwthiker/curl-impersonate/issues/207#issuecomment-1809859159

nklido avatar Nov 14 '23 09:11 nklido

Same issue and it affects https://github.com/RSS-Bridge/rss-bridge project.

What I found is that the version returned by print_r(curl_version()); is of the actual libcurl installed in the system (7.88), despite the LD_PRELOAD used, so somehow PHP ignores the dynamically-linked curl-impersonate (at 7.84).

According to your readme,

The CURL_IMPERSONATE env var has two effects:

curl_easy_impersonate() is called automatically for any new curl handle created by curl_easy_init().

So I checked with gdb and despite the curl-impersonate so lib getting loaded as expected, the curl_easy_impersonate() is not getting executed by curl_easy_init():

root@64298d0fb1ee:/# gdb --args php -r '$ch = curl_init("https://testurl.com"); curl_exec($ch);'
GNU gdb (Debian 13.1-3) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from php...
(No debugging symbols found in php)
(gdb) set environment LD_PRELOAD /usr/local/lib/curl-impersonate/libcurl-impersonate-ff.so
(gdb) set environment CURL_IMPERSONATE ff102
(gdb) set stop-on-solib-events 1
(gdb) break curl_easy_init
Function "curl_easy_init" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (curl_easy_init) pending.
(gdb) break curl_easy_impersonate
Function "curl_easy_impersonate" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 2 (curl_easy_impersonate) pending.
(gdb) start
Function "main" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Temporary breakpoint 3 (-qualified main if $_inferior == 1) pending.
Starting program: /usr/bin/php -r \$ch\ =\ curl_init\(\"https://testurl.com"\)\;\ curl_exec\(\$ch\)\;
warning: Error disabling address space randomization: Function not implemented
Stopped due to shared library event (no libraries added or removed)
(gdb) c
Continuing.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Stopped due to shared library event:
  Inferior loaded /usr/local/lib/curl-impersonate/libcurl-impersonate-ff.so
    /lib/x86_64-linux-gnu/libm.so.6
    /lib/x86_64-linux-gnu/libxml2.so.2
    /lib/x86_64-linux-gnu/libssl.so.3
    /lib/x86_64-linux-gnu/libcrypto.so.3
    /lib/x86_64-linux-gnu/libpcre2-8.so.0
    /lib/x86_64-linux-gnu/libz.so.1
    /lib/x86_64-linux-gnu/libsodium.so.23
    /lib/x86_64-linux-gnu/libargon2.so.1
    /lib/x86_64-linux-gnu/libc.so.6
    /lib/x86_64-linux-gnu/libdl.so.2
    /lib/x86_64-linux-gnu/libpthread.so.0
    /lib/x86_64-linux-gnu/libicuuc.so.72
    /lib/x86_64-linux-gnu/liblzma.so.5
    /lib/x86_64-linux-gnu/libicudata.so.72
    /lib/x86_64-linux-gnu/libstdc++.so.6
    /lib/x86_64-linux-gnu/libgcc_s.so.1
(gdb) c

[...]

(gdb) c
Continuing.
Stopped due to shared library event (no libraries added or removed)
(gdb) c
Continuing.
Stopped due to shared library event:
  Inferior loaded /usr/lib/php/20220829/curl.so
(gdb) c
Continuing.
Stopped due to shared library event:
  Inferior loaded /lib/x86_64-linux-gnu/libcurl.so.4
    /lib/x86_64-linux-gnu/libnghttp2.so.14
    /lib/x86_64-linux-gnu/libidn2.so.0
    /lib/x86_64-linux-gnu/librtmp.so.1
    /lib/x86_64-linux-gnu/libssh2.so.1
    /lib/x86_64-linux-gnu/libpsl.so.5
    /lib/x86_64-linux-gnu/libgssapi_krb5.so.2
    /lib/x86_64-linux-gnu/libldap-2.5.so.0
    /lib/x86_64-linux-gnu/liblber-2.5.so.0
    /lib/x86_64-linux-gnu/libzstd.so.1
    /lib/x86_64-linux-gnu/libbrotlidec.so.1
    /lib/x86_64-linux-gnu/libunistring.so.2
    /lib/x86_64-linux-gnu/libgnutls.so.30
    /lib/x86_64-linux-gnu/libhogweed.so.6
    /lib/x86_64-linux-gnu/libnettle.so.8
    /lib/x86_64-linux-gnu/libgmp.so.10
    /lib/x86_64-linux-gnu/libkrb5.so.3
    /lib/x86_64-linux-gnu/libk5crypto.so.3
    /lib/x86_64-linux-gnu/libcom_err.so.2
    /lib/x86_64-linux-gnu/libkrb5support.so.0
    /lib/x86_64-linux-gnu/libsasl2.so.2
    /lib/x86_64-linux-gnu/libbrotlicommon.so.1
    /lib/x86_64-linux-gnu/libp11-kit.so.0
    /lib/x86_64-linux-gnu/libtasn1.so.6
    /lib/x86_64-linux-gnu/libkeyutils.so.1
    /lib/x86_64-linux-gnu/libresolv.so.2
    /lib/x86_64-linux-gnu/libffi.so.8
(gdb) set stop-on-solib-events 0
(gdb) c
Continuing.

Breakpoint 1.1, 0x00007efdd0774e10 in curl_easy_init () from /lib/x86_64-linux-gnu/libcurl.so.4
(gdb) c
Continuing.

Breakpoint 1.1, 0x00007efdd0774e10 in curl_easy_init () from /lib/x86_64-linux-gnu/libcurl.so.4
(gdb) c
Continuing.
[New Thread 0x7efdcea406c0 (LWP 944)]
[Thread 0x7efdcea406c0 (LWP 944) exited]
<html><head><title>TESTURL</title></head><body style="margin:0"><p id="cmsg">Please enable JS and disable any ad blocker</p></body></html>
warning: Temporarily disabling breakpoints for unloaded shared library "/lib/x86_64-linux-gnu/libcurl.so.4"
[Inferior 1 (process 941) exited normally]

So the question why is that? In any case, I don't think PHP does anything specific here, it appears that LD_PRELOAD trick stopped working at some point?

kultigspritzig avatar Jan 12 '24 14:01 kultigspritzig

I just tested with php-8.1 on Ubuntu 22.04 with curl-impersonate 0.5.4, it worked. What is your system infomation?

╰─>$ LD_PRELOAD=./libcurl-impersonate-chrome.so CURL_IMPERSONATE=chrome101 php -r 'print_r(curl_version());'
Array
(
    [version_number] => 480256
    [age] => 9
    [features] => 1370063517
    [ssl_version_number] => 0
    [version] => 7.84.0
    [host] => x86_64-pc-linux-gnu
    [ssl_version] => BoringSSL

perkfly avatar Jan 12 '24 16:01 perkfly

Testing this in a debian:12-slim docker container, since this in turn is the basis of rssbridge container.

kultigspritzig avatar Jan 12 '24 17:01 kultigspritzig

I double tested and the LD_PRELOAD simply doesn't work at all in this setup. What works is directly symlinking curl-impersonate so lib to libcurl.

wrobelda avatar Feb 24 '24 21:02 wrobelda