TLS through Proxy raises unexpected TLS errors.

[ereOn]

Bug Report

Fluent-bit can't send HTTPS messages through a HTTP proxy.

• Version used: fluent-bit v1.0.6
• Configuration:

[OUTPUT]
    Name http
    Match *
    Host fluentd-host-somewhere.com
    Port 443
    URI /index
    Format json
    tls On
    HTTP_User myuser
    HTTP_Passwd mypass
    Proxy http://proxy:3128

Here is the error message I'm getting (repeats for each sent batch):

[2019/05/14 21:35:44] [error] [io_tls] flb_io_tls.c:304 SSL - An invalid SSL record was received
[2019/05/14 21:35:44] [error] [out_http] no upstream connections available to proxy:3128

We have no choice but to go through the corporate proxy.

Note that this is not about using a HTTPS proxy. The proxy is HTTP, but the underlying connection is HTTPS.

I would expect a CONNECT to happen on the proxy.

We also tried removing the tls On option in the configuration but this causes fluent-bit to crash (SIGSEV) on this line.

[cwiggs]

I ran into this issue recently while trying to use fluentbit with AWS firelens. The workaround for me was to just use fluentd, which seems to issue the CONNECT properly.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.

[hankwallace]

I have this same issue using aws-for-fluent-bit and firelens. Most of the firelens containers start and operate correctly for a time, but then I see these errors in the firelens log:

2022-08-11T22:04:40.933-04:00	[2022/08/12 02:04:40] [error] [tls] error: unexpected EOF
2022-08-11T22:04:40.933-04:00	[2022/08/12 02:04:40] [debug] [upstream] connection #75 failed to <loki-url>:443
2022-08-11T22:04:40.933-04:00	[2022/08/12 02:04:40] [error] [output:loki:loki.1] no upstream connections available

[epsteina16]

Reopening this issue since we have run into it while trying to configure fluent bit to forward logs to datadog over TLS via a HTTP proxy. The issue is in the core HTTP code (Fluent Bit does not send a CONNECT message) and as a result will affect all output plugins that use HTTP/HTTPS.

This is an important issue for us, and given that HTTP proxy support is currently well-documented despite these issues (https://docs.fluentbit.io/manual/administration/http-proxy), it would be great to see this prioritized.

[epsteina16]

Any updates on this? @leonardo-albertovich would it be possible for you or someone else who has worked on the core HTTP code to take a look at this?

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[hankwallace]

Any updates on this? The issue is still causing problems in our environments.

[agup006]

There is new work underway to introduce a setting net.proxy that works for all output plugins, most likely in the next quarter