Http Client - connections not reused

[scottlepp]

What happened: Using httpClient.New(opts) creates a new Transport every time and therefore will not reuse http connections. This caused a problem with Salesforce on Grafana Cloud having too many open http connections.

What you expected to happen: Reuse connections.

How to reproduce it (as minimally and precisely as possible): Use httpClient.New(options) with options defined. Trace requests like this and log if the connection is reused.

Anything else we need to know?: Workaround is to use httpClient.New() which just returns the Default client (which makes using this client not very useful)

@marefr

[wbrowne]

@scottlepp Apologies if I'm not following correctly, but is there a reason why we can't create the HTTP client outside of the InstanceFactoryFunc? Then SFClient could for example have the HTTP client as field, and then adjust that if necessary from the InstanceFactoryFunc

[scottlepp]

@wbrowne Yes we can, though if we don't know to do that and follow a typical pattern where the http client is created on every request, then we unknowingly create a client that doesn't reuse connections ( as I did ) and caused problems on our cloud env.

[marefr]

@scottlepp I think I see what's wrong with your code. You're creating new HTTP clients in each SFClient methods, e.g. https://github.com/grafana/salesforce-datasource/blob/b66f68a0d12810baa8d18c4b5d0d659cdab94b6d/pkg/client.go#L87.

With the connection settings here https://github.com/grafana/salesforce-datasource/blob/b66f68a0d12810baa8d18c4b5d0d659cdab94b6d/pkg/client.go#L46

It should either

• require a HTTP client to be included in the connection settings and assign it to a field of SFClient struct
• instantiate the HTTP client and assign it to a field of SFClient struct

Then reuse the SFClient field in each method.

Your InstanceFactoryFunc named NewInstance will be called whenever a new datasource is created or an existing one in updated in Grafana so that's a great opportunity to instantiate a HTTP client, e.g. https://github.com/grafana/salesforce-datasource/blob/b66f68a0d12810baa8d18c4b5d0d659cdab94b6d/pkg/main.go#L34

Does this make sense?