hive-solr icon indicating copy to clipboard operation
hive-solr copied to clipboard

Sockect Read timeout exception while commiting to solr

Open tvinod06 opened this issue 3 years ago • 0 comments

Hi

I'm pushing 600 million records every day to solr using Hive. As data is huge, solr is responding slow. Due to which i'm getting socket timeout exception. Read timeout.

Can anyone help me how to increase sockettimeout ? is it configurable ?

Help is much appreciated. Thanks in advance.

Caused by: shaded.org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://{hostname}:8983/solr/{collection} at shaded.org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654) at shaded.org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255) at shaded.org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) at shaded.org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483) at shaded.org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413) at shaded.org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106) at shaded.org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886) at shaded.org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819) at shaded.org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194) at shaded.org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484) at shaded.org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:501) at com.lucidworks.hadoop.io.LucidWorksWriter.close(LucidWorksWriter.java:278) ... 23 more Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at shaded.org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) at shaded.org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) at shaded.org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282) at shaded.org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at shaded.org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at shaded.org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) at shaded.org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) at shaded.org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at shaded.org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) at shaded.org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) at shaded.org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at shaded.org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at shaded.org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at shaded.org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at shaded.org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at shaded.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at shaded.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) at shaded.org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)

tvinod06 avatar Feb 22 '22 17:02 tvinod06