ziglang
error: TlsInitializationFailed for std.http.Client.fetch with http-URL
Zig Version
0.14.0-dev.1820+ea527f7a8
Steps to Reproduce and Observed Behavior
_ = try std.http.Client.fetch(&client, .{
.location = .{
.url = "http://wttr.in/pdx?format=%t",
},
.response_storage = std.http.Client.FetchOptions.ResponseStorage{
.dynamic = &r,
},
});
leads to:
error: TlsInitializationFailed
/home/michael/opt/zig-linux-x86_64-0.14.0-dev.1820+ea527f7a8/lib/std/crypto/tls/Client.zig:568:49: 0x124fddb in init__anon_21743 (zstatus)
.certificate => return error.TlsCertificateNotVerified,
^
/home/michael/opt/zig-linux-x86_64-0.14.0-dev.1820+ea527f7a8/lib/std/http/Client.zig:1357:99: 0x118a07f in connectTcp (zstatus)
conn.data.tls_client.* = std.crypto.tls.Client.init(stream, client.ca_bundle, host) catch return error.TlsInitializationFailed;
^
/home/michael/opt/zig-linux-x86_64-0.14.0-dev.1820+ea527f7a8/lib/std/http/Client.zig:1492:14: 0x1153ee0 in connect (zstatus)
} orelse return client.connectTcp(host, port, protocol);
^
/home/michael/opt/zig-linux-x86_64-0.14.0-dev.1820+ea527f7a8/lib/std/http/Client.zig:808:26: 0x1158ab5 in redirect (zstatus)
req.connection = try req.client.connect(new_host, uriPort(valid_uri, protocol), protocol);
^
/home/michael/opt/zig-linux-x86_64-0.14.0-dev.1820+ea527f7a8/lib/std/http/Client.zig:1061:17: 0x1130a66 in wait (zstatus)
try req.redirect(req.uri.resolve_inplace(
^
/home/michael/opt/zig-linux-x86_64-0.14.0-dev.1820+ea527f7a8/lib/std/http/Client.zig:1742:5: 0x112b54b in fetch (zstatus)
try req.wait();
^
/home/michael/projects/zstatus/src/main.zig:22:17: 0x114b376 in main (zstatus)
_ = try std.http.Client.fetch(&client, .{
^
std.http.Client.fetch with https-URL instead of http-URL works
curl / wget can fetch from http-URL
when i tested with curl / wget i could not see any redirect from http to https for the test url, no tls involved there
Expected Behavior
I expect zig not to do any tls when i fetch a http-URL
The reason you don't see the HTTPS redirect when using curl or wget is because wttr.in has hard-coded certain user agents as "plain text" and will not apply the HTTPS redirect to them: https://github.com/chubin/wttr.in/blob/235581925fa2e09a42e9631e2c23294a1972ee0e/internal/processor/processor.go#L25-L41
Simulating a request from curl using Zig's user-agent header can reproduce this behavior:
$ curl -H 'User-Agent: zig/0.13.0 (std.http)' -v 'http://wttr.in/pdx?format=%t'
* Host wttr.in:80 was resolved.
* IPv6: (none)
* IPv4: 5.9.243.187
* Trying 5.9.243.187:80...
* Connected to wttr.in (5.9.243.187) port 80
> GET /pdx?format=%t HTTP/1.1
> Host: wttr.in
> Accept: */*
> User-Agent: zig/0.13.0 (std.http)
>
* Request completely sent off
< HTTP/1.1 301 Moved Permanently
< Access-Control-Allow-Origin: *
< Location: https://wttr.in/pdx?format=%t
< Date: Wed, 09 Oct 2024 01:27:36 GMT
< Content-Length: 224
< Content-Type: text/html; charset=utf-8
<
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="https://wttr.in/pdx?format=%t">here</A>.
</BODY></HTML>
* Connection #0 to host wttr.in left intact
You should be able to work around this in your Zig program by overriding the user-agent header with one containing one of the aforementioned "plain text" user agent signatures (or make a PR to wttr.in to add Zig's user agent to their "plain text" list).
ty for finding the evil here :)
then again its still a bug or missing feature in zig stdlib. a redirect from http to https should enable zig to speak tls to the new redirect https url.
a redirect from http to https should enable zig to speak tls to the new redirect https url.
The error return trace points out the specific issue, Zig is speaking TLS to the HTTPS url. You were redirected, and the certificate provided by the HTTPS server could not be verified using the default root CA chain.
There is something that could be improved here: due to #14340, the root CA certificates are lazily scanned in std.http.Client.open: https://github.com/ziglang/zig/blob/bd8ef0036d8d380a753bb88d426f58c059c97f61/lib/std/http/Client.zig#L1626-L1637 but this logic is not executed when handling a redirect, so redirecting from HTTP to HTTPS on a client which has not made any HTTPS requests yet does not work.
I have a commit on my fork which just moves the lazy scan logic into connectTcp right before initializing the TLS client: https://github.com/ianprime0509/zig/tree/https-redirect-init However, given #14171 has not been implemented, it's not possible to add any tests for this yet, so per the general direction given in https://github.com/ziglang/zig/pull/18507#issuecomment-2103587862, I'm going to hold off on a PR for that for now unless one is desired without test coverage.