Too many files open error due to connections in close_wait state while handling a usecase processing lots of data

[manmohanpv]

Too many files open error due to connections in close_wait state while handling a usecase processing lots of data. Any idea?

fetchData method gets called 10,000 times sequentially in some cases and got too many files open state due to close wait connections exceed the system limit for total open files.

Any idea why RetryTemplate not closing the connection properly here? Should i set connection, close header parameter to avoid this case? Do we have any connection pooling mechanism available in RetryTemplate or RestTemplate?

public Info[] fetchData(Integer rId) throws Exception {
	Info[] Info = null;
	if (rId != null) {
		try {
			HttpHeaders headers = new HttpHeaders();
			headers.setAll(reqHelper.getAllJsonBasedRequestHeaders());
			HttpEntity<String> entity = new HttpEntity<String>("parameters", headers);
			ResponseEntity<Info[]> response = getRetryTemplate()
					.execute(new RetryCallback<ResponseEntity<Info[]>, Exception>() {
						@Override
						public ResponseEntity<Info[]> doWithRetry(RetryContext context) throws Exception {
							return getRestTemplate().exchange(url),
									HttpMethod.GET, entity, Info[].class);
						}
					});
			if (response.getStatusCode() == HttpStatus.OK) {
				Info = response.getBody();
			}
		} catch (Exception e) {

			throw e;
		}
	}
	return Info;
}



private RetryTemplate getRetryTemplate() {
	FixedBackOffPolicy backOffPolicy = new FixedBackOffPolicy();
	backOffPolicy.setBackOffPeriod(serviceConfig.getTimeToBackOff());
	RetryTemplate template = new RetryTemplate();
	template.setRetryPolicy(apiRetryPolicy);
	template.setBackOffPolicy(backOffPolicy);
	return template;
}




apiRetryPolicy:

public void init()
{
    final SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
    simpleRetryPolicy.setMaxAttempts(radarServiceProperties.getMaxRetryAttempts());
    this.setExceptionClassifier( new Classifier<Throwable, RetryPolicy>()
    {
        @Override
        public RetryPolicy classify( Throwable classifiable )
        {
                if ( classifiable instanceof ConnectException 
                		|| classifiable instanceof NullPointerException 
                		|| classifiable instanceof UnknownHostException)
                {
                        return new NeverRetryPolicy();
                }
                
                if(classifiable instanceof HttpStatusCodeException )
                {
                	HttpStatus errorCode = ((HttpStatusCodeException) classifiable).getStatusCode();
                	if(errorCode == HttpStatus.UNAUTHORIZED)
                	{
                		 return new NeverRetryPolicy();
                	}
                }

                return simpleRetryPolicy;
            }
        } );
    }

}