cpr icon indicating copy to clipboard operation
cpr copied to clipboard

INTERNAL_ERROR(4) on any requests that goes to a site protected with CloudFlare

Open veteri opened this issue 1 year ago • 3 comments

Description

If you perform a simple get requests to any site that isnt using CloudFlare, everything is fine and works as expected. However if you do the same with any site using cloudflare, the response has a status_code of 0 and INTERNAL_ERROR(4).

Example/How to Reproduce

This will result in the above mentioned INTERNAL_ERROR

cpr::Url url{ "https://discord.com" };
cpr::Session session;
session.SetOption(url);

cpr::Response r = session.Get();

std::cout << "Status code: " << r.status_code << '\n';
std::cout << "Header:\n";
if (r.status_code == 0) {

	std::cout << "Error code: " << r.error.message << '\n';
}
for (const std::pair<std::string, std::string>& kv : r.header) {
	std::cout << '\t' << kv.first << ':' << kv.second << '\n';
}
std::cout << "Text: " << r.text << '\n';

This will work fine:

cpr::Url url{ "https://google.com" };
cpr::Session session;
session.SetOption(url);

cpr::Response r = session.Get();

std::cout << "Status code: " << r.status_code << '\n';
std::cout << "Header:\n";
if (r.status_code == 0) {

	std::cout << "Error code: " << r.error.message << '\n';
}
for (const std::pair<std::string, std::string>& kv : r.header) {
	std::cout << '\t' << kv.first << ':' << kv.second << '\n';
}
std::cout << "Text: " << r.text << '\n';

Possible Fix

No response

Where did you get it from?

vcpkg

Additional Context/Your Environment

  • OS: Windows
  • Version: 10
  • CPR (vcpkg): 1.10.5#2
  • curl (vcpkg): 8.8.0#2

veteri avatar Jun 25 '24 15:06 veteri

@veteri thanks for reporting. I will try to look into it over the weekend.

COM8 avatar Jun 27 '24 19:06 COM8

I faced the same issue with a cmake+vcpkg project,

It worked fine after downgrading to version 1.9.3

vcpkg.json:

{ "name": "app", "version-string": "1.0.0", "dependencies": [ { "name": "nlohmann-json" }, { "name": "fmt" }, { "name": "spdlog" }, { "name": "sdl2" }, { "name": "cpr" }, { "name": "ixwebsocket" }, { "name": "concurrentqueue" }, { "name": "sdl2-mixer" }, { "name": "cryptopp" }, { "name": "sqlite3" }, { "name": "openssl" }, { "name": "opengl" }, { "name": "boost-algorithm" } ], "overrides": [ { "name": "cpr", "version": "1.9.3" } ], "builtin-baseline": "04b0cf2b3fd1752d3c3db969cbc10ba0a4613cee" }

prabuinet avatar Jun 27 '24 20:06 prabuinet

I can not confirm this issue. For me on Linux (Fedora) with all combinations (cpr master, 1.10.5 and curl 8.4.0 and 8.8.0) it works. I'm not using vcpkg. vcpkg is not maintained by us so, I don't know how they do it.

That's what I used: https://github.com/libcpr/example-cmake-fetch-content

COM8 avatar Jun 29 '24 12:06 COM8

Same issue here. Cloudflare changed something with their new certificates and public key pin not working anymore. Cpr from vcpkg is broken too.

Do this to domain with new cloudflare cert and you will get s_client -connect my_domain_with_new_cloudflare_cert:443 | openssl x509 -pubkey -noout | openssl rsa -pubin -outform der | openssl dgst -sha256 -binary | openssl enc -base64

depth=2 C=US, O=Internet Security Research Group, CN=ISRG Root X1 verify error:num=20:unable to get local issuer certificate verify return:1 depth=1 C=US, O=Let's Encrypt, CN=E6 verify return:1 depth=0 CN=******************** verify return:1 Not an RSA key 47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=

Verbose:

  • Connected to ******************************** port 443
  • schannel: disabled automatic use of client certificate
  • schannel: connection hostname () validated against certificate name ()
  • Subject: CN=********************************
  • Issuer: C=US, O=Let's Encrypt, CN=E6
  • Version: 2
  • Serial Number: 03:cf:f5:40:86:7e:2e:f9:b6:d2:e6:24:17:ff:eb:b9:58:c9:
  • Closing connection
  • schannel: shutting down SSL/TLS connection with t******************************** port 443

Even tried all these opts: cpr::ssl::VerifyPeer peer{ false }; cpr::ssl::VerifyHost host{ false }; cpr::ssl::ALPN alpn{ false }; cpr::ssl::VerifyStatus verifyStatus{ false }; cpr::Verbose opt3{ true };

cpr::SslOptions sslOpts = cpr::Ssl( peer, host, alpn, verifyStatus );

Have not tried moving to non-vcpkg version yet.

Has someone tried reporting this to vcpkg maintainers and got any response?

Chrys4lisfag avatar Jul 05 '24 11:07 Chrys4lisfag

I can not confirm this issue. For me on Linux (Fedora) with all combinations (cpr master, 1.10.5 and curl 8.4.0 and 8.8.0) it works. I'm not using vcpkg. vcpkg is not maintained by us so, I don't know how they do it.

Has somebody tested with openssl on Windows? On Linux, it wouldn't use schannel.

dg0yt avatar Jul 08 '24 04:07 dg0yt

I can not confirm this issue. For me on Linux (Fedora) with all combinations (cpr master, 1.10.5 and curl 8.4.0 and 8.8.0) it works. I'm not using vcpkg. vcpkg is not maintained by us so, I don't know how they do it.

Has somebody tested with openssl on Windows? On Linux, it wouldn't use schannel.

No, haven't tested with openssl backend yet

Chrys4lisfag avatar Jul 08 '24 09:07 Chrys4lisfag

btw, regarding the problem with public key pin, cloudflare told me that this feature is not supported by new certificates anymore, so that is not the library issue, but this change broke https somehow for windows vcpkg version

Chrys4lisfag avatar Jul 08 '24 11:07 Chrys4lisfag

Since it is no cpr issue, I'm closing this one.

COM8 avatar Jul 10 '24 06:07 COM8

I'm confused as to how it's not a cpr issue. Using a previous version of cpr works perfectly fine, just the latest version has this problem. So it's definitely somehow related to the update. I'm also a bit confused about you not knowing about the vcpkg version? Didn't you have to explicitly ask to get the lib on the registry yourself or who was the person doing so?

For anyone still wanting to use cpr on windows, which i assume is a significant amount of people what worked for me is forcing vcpkg to use the older version version prior to the update that broke everything, put this into your vcpkg.json and enable manifest mode if you havent already:

{
    "name": "your-project-name",
    "version": "1.0.0",
    "dependencies": [
        "cpr",
    ],
    "builtin-baseline": "6ab331448c76fbd1ca4fb599d374924e6d5e3024"
}

veteri avatar Sep 18 '24 14:09 veteri

@veteri it's related to a curl update (Ref) done by vcpkg.

Also I'm aware of the vcpkg version but I neither maintain it nor did I initially create it. It's a community effort.

Why do I not maintain the vcpkg version? Because of time. I don't have time for it right now. I'm not a Windows user and I also do not know how vcpkg works.

Since I'm doing this in my free time without getting paid by anyone for my work I decided that I (for now) can not effort to maintain the vcpkg port.

COM8 avatar Sep 20 '24 15:09 COM8

@veteri it's related to a curl update (Ref) done by vcpkg.

That ref points to a comment which claims that the issue would be solved with newer versions of libcurl. vcpkg has the newest version of libcurl (8.10.1).

Why do I not maintain the vcpkg version? Because of time. I don't have time for it right now. I'm not a Windows user and I also do not know how vcpkg works.

This is okay. If a project has a proper build system, maintenance of packages can be done by others. Package maintainers may also have a different perspective on how to align installations within the package manager's ecosystem.

Note that vcpkg is not Windows-only.

dg0yt avatar Sep 21 '24 05:09 dg0yt

Put this into your vcpkg.json and enable manifest mode if you havent already:

@veteri I would disagree. With builtin-baseline, you pin every package to old versions. But according to the discussion, you only need to control the version of curl. This is a case for per-port overrides.

dg0yt avatar Sep 21 '24 05:09 dg0yt