elastic icon indicating copy to clipboard operation
elastic copied to clipboard

Post <"elatic_url"> read tcp <ip1>-><ip2>: read: connection timed out

Open avi94 opened this issue 3 years ago • 4 comments

Please use the following questions as a guideline to help me answer your issue/question without further inquiry. Thank you.

Which version of Elastic are you using?

[v ] elastic.v7 (for Elasticsearch 7.x) [ ] elastic.v6 (for Elasticsearch 6.x) [ ] elastic.v5 (for Elasticsearch 5.x) [ ] elastic.v3 (for Elasticsearch 2.x) [ ] elastic.v2 (for Elasticsearch 1.x)

Please describe the expected behavior

We use elasticsearch in our backend, and have been for a long time. We expect response from ES.

Please describe the actual behavior

Quite often we see this kind of error: Post <"elatic_url"> read tcp ->: read: connection timed out. Can someone guide me on why this error happens or what this error means?

Any steps to reproduce the behavior?

The error looks like this: Post "<elastic_url>/index/_count": read tcp ip1:port1->ip2:port2: read: connection timed out

We encountered it this morning and about 150 such requests failed. The requests were to Count API, Search API, Delete_By_Query API etc. Nothing specifically was the problem. All ES endpoints were throwing these errors.

Usually this does not happen and I understand this is an Elasticsearch problem rather than a problem with this library, but would it be possible to help on why this stuff happens? And is there a way to prevent this? Also how does timeouts work in ES?

Thanks in advance!

avi94 avatar Jan 29 '22 07:01 avi94

I've not seen this yet. Looks like an underlying networking issue. Is it somehow related to #1585?

olivere avatar Feb 17 '22 12:02 olivere

@olivere it could well be related to #1585, but I'm not quite sure if it is. Meanwhile, it would be extremely helpful if you could kindly let me know how timeouts work in elasticsearch, and specifically how timeouts work with this particular library. I hear different APIs have different timeout settings, but how does this library configure timeouts? Please help me on this. Thanks in advance!

avi94 avatar Feb 24 '22 09:02 avi94

@avi94 This library doesn't do anything special to timeouts. Three aspects come to mind when thinking about this:

  1. Retries (configured on the client-level)
  2. Configuring the *http.Client to use (you can define your own)
  3. Timeouts as allowed by specific ElasticSearch APIs (e.g. there is master_timeout and all sorts of API-specific timeouts). Most basically control/hint how ES should operate on the server-side.

Im general I tried very hard to keep the library as dumb as possible.

olivere avatar Feb 24 '22 12:02 olivere

Ok thank you @olivere I will look into the points you mentioned.

avi94 avatar Feb 28 '22 03:02 avi94