aws: fix when not running in EC2 and having an HTTP proxy

[Sh4d1]

This patch fix the case where fluent-bit is not running in EC2 but is running with the HTTP_PROXY variable set.

In this case the flb_ec2_provider_create function will fail since the check on the tcp_host and tcp_port in the flb_aws_imds_create function will fail and return NULL. (Beause it is changed here)

Signed-off-by: Patrik Cyvoct [email protected]

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing Before we can approve your change; please submit the following in a comment:

• [N/A] Example configuration file for the change
• [N/A] Debug log output from testing the change

• [N/A] Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• [N/A] Attached local packaging test output showing all targets (including any new ones) build.

Documentation

• [N/A] Documentation required for this feature

Backporting

• [ ] Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[PettitWesley]

@Sh4d1 Do you think anyone would ever want to connect to IMDS over a proxy? I think no... that seems silly and unsafe.

If you have a proxy configured though, then I think you would still want to be able to run on EC2.

@matthewfala The problem is your code here I think: https://github.com/fluent/fluent-bit/blob/master/src/aws/flb_aws_imds.c#L84

Which now conflicts with this proxy code: https://github.com/fluent/fluent-bit/blob/master/src/flb_upstream.c#L256

Do we truly NEED to validate that the host and port like this? Seems unnecessary.

(See PR description)

[Sh4d1]

@PettitWesley

Do you think anyone would ever want to connect to IMDS over a proxy? I think no... that seems silly and unsafe.

Nope but since the client gets initialized anyway, it cannot currently be used with an HTTP proxy

[matthewfala]

The check was originally added for developers out of concern that they might give the IMDS client some misconfigured upstream. I didn't realize that the proxy settings could alter the host on this upstream.

@PettitWesley I think that IMDS should not go through proxy ever right? So the check may be valid, in caching IMDS use through proxy. I don't really understand how the proxy settings work, but I would imagine that the solution would be turning off proxy for IMDS connections if possible. What do you think?

[PettitWesley]

@matthewfala I think we should probably remove the check you added... right now with how the proxying works, the user has to set IMDS in the no_proxy settings: https://github.com/fluent/fluent-bit/blob/master/src/flb_upstream.c#L245

@edsiper @leonardo-albertovich I think the solution here is that plugins need to be able to create upstreams that don't ever use a proxy. Basically, our code can say create a connection but ignore the proxy settings. It looks like we could do this right now by manually overriding the host and port, but it'd be better to just contribute a new create function or change the existing signature I think.

[edsiper]

@PettitWesley yeah, some kind of flb_upstream_no_proxy() .

is there any action pending on my side here ?

[PettitWesley]

is there any action pending on my side here ?

@edsiper I don't think so. We can submit the PR.

[PettitWesley]

I think the solution here is to refactor this code into 2 major functions: https://github.com/fluent/fluent-bit/blob/master/src/flb_upstream.c#L222

• flb_upstream_create: existing functionality that contains proxy settings
• flb_upstream_create_no_proxy: same thing but without all of the proxy code.

The best way to do this is probably to have a common helper function that both can call out too.

@Sh4d1 I am curious if you are comfortable implementing this fix instead of your current fix, or if instead I should ask @matthewfala for it.

[github-actions[bot]]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[Sh4d1]

Sorry @PettitWesley, completly forgot to answer :sweat_smile: I'm not using fluentbit anymore, and don't really have the time to bump my PR

[github-actions[bot]]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.