Increase Error and Logging Verbosity for Connections

[RoseSecurity]

Why:

• Encountered a connection error with Elasticsearch where I accidentally attempted to connect over HTTP rather than HTTPS. Received a blanket "EOF" error each time

What:

• Would it be possible to add more logging to internal/clients/config/elasticsearch.go?

Problem:

I have traced the lack of verbosity to an antiquated function in internal/clients/config/debug.go. The function makes use of estransport which has been deprecated.

var _ estransport.Logger = &debugLogger{}

type debugLogger struct {
	Name string
}

func (l *debugLogger) LogRoundTrip(req *http.Request, resp *http.Response, err error, start time.Time, duration time.Duration) error {
	ctx := req.Context()
	requestId := "<nil>"
	if req != nil {
		requestId = fmt.Sprintf("%s %s", req.Method, req.URL)
	}
	tflog.Debug(ctx, fmt.Sprintf("%s request [%s] executed. Took %s. %#v", l.Name, requestId, duration, err))

	if req != nil && req.Body != nil {
		l.logRequest(ctx, req, requestId)
	}

	if resp != nil && resp.Body != nil {
		l.logResponse(ctx, resp, requestId)
	}

	if resp == nil {
		tflog.Debug(ctx, fmt.Sprintf("%s response for [%s] is nil", l.Name, requestId))
	}

	return nil
}

I suggest that we move to: https://github.com/elastic/elastic-transport-go

[Honken77]

I have a similar issue now where I only get an EOF error. The URL is correct, so it's probably something in the payload for my elasticstack_kibana_alerting_rule but I have no idea what.

[RoseSecurity]

@Honken77 I think adding more error validation is definitely essential. I can take a deeper look and open a PR but I don't want to step on any toes.

[tobio]

I can take a deeper look and open a PR but I don't want to step on any toes.

There's no toes here, we'd love to see any contributions you're able to make here. Thanks!

[RoseSecurity]

@tobio I'm not extremely familiar with the Elasticsearch Go client, but can you confirm that the estransport package has been deprecated?

[tobio]

@RoseSecurity estransport has been moved to https://github.com/elastic/elastic-transport-go in the 8.x client series. The provider maintains compatibility with 7.17 for the time being so we've not moved the client forward at this point.

That said, the Logger interface looks to be equivalent between the two so any implementations of elastictransport.Logger within https://github.com/elastic/elastic-transport-go would also implement the estransport.Logger interface.

[RoseSecurity]

@RoseSecurity estransport has been moved to https://github.com/elastic/elastic-transport-go in the 8.x client series. The provider maintains compatibility with 7.17 for the time being so we've not moved the client forward at this point.

That said, the Logger interface looks to be equivalent between the two so any implementations of elastictransport.Logger within https://github.com/elastic/elastic-transport-go would also implement the estransport.Logger interface.

Thank you for the backstory! I have a draft PR open with an update to 8.X but if the plan isn't to migrate the other 7.X dependencies yet, I can close it. It's difficult because if Elasticsearch responds with an empty response, not much can be gleaned when it comes to troubleshooting the cause. Don't know a good way to approach it