WebClientEurekaHttpClient + cloud config is leaking file descriptors

[johnny]

We noticed one of our minimal webflux based services was leaking file descriptors every 30 seconds

...
java    23973  xx  539u     sock                0,8       0t0 277263 protocol: TCPv6
java    23973  xx  540u     sock                0,8       0t0 277327 protocol: TCPv6
java    23973  xx  541u     sock                0,8       0t0 277392 protocol: TCPv6
java    23973  xx  542u     sock                0,8       0t0 277454 protocol: TCPv6
java    23973  xx  543u     sock                0,8       0t0 277516 protocol: TCPv6
...

This didn't occur with our pure webmvc based services. We traced it down to the following sequence

1. Every 30 seconds, a HeartbeatEvent is created, triggering a refresh of the config service instance list through org.springframework.cloud.config.client.DiscoveryClientConfigServiceBootstrapConfiguration.HeartbeatListener
2. This calls org.springframework.cloud.netflix.eureka.config.EurekaConfigServerBootstrapConfiguration#eurekaConfigServerInstanceProvider which in turn uses the reactive web client through a blocking interface (com.netflix.discovery.shared.transport.EurekaHttpClient#getApplications)
3. Within org.springframework.cloud.netflix.eureka.http.WebClientEurekaHttpClient#getApplicationsInternal occurs a call to .block() which leaks the connection

We worked around this by disabling the heartbeat listener.

@Configuration
@ConditionalOnBean(name = ["heartbeatListener"])
class EurekaHeartbeatFix {
    @Autowired
    private lateinit var context: ApplicationContext

    @Autowired
    private lateinit var heartbeatListener: SmartApplicationListener

    @PostConstruct
    fun removeEurekaHeartbeatListener() {
        val parent = context.parent?.getBean(
            AbstractApplicationContext.APPLICATION_EVENT_MULTICASTER_BEAN_NAME,
            ApplicationEventMulticaster::class.java
        )

        parent?.removeApplicationListenerBean("heartbeatListener")
        parent?.removeApplicationListener(heartbeatListener)
    }
}

Library Versions: spring-cloud-netflix-eureka-client:2.2.3.RELEASE spring-cloud-config-client:2.2.3.RELEASE OS: Linux

[spencergibb]

Version 2.2.4.RELEASE make WebClient opt in so you shouldn't need that work around anymore.

@rstoyanchev does any of this sound familiar that calling exchange().block() on a WebClient call would leak a connection?

[rstoyanchev]

It shouldn't but with exchange() you are responsible to handle all possible response types. From a quick look, I see handling for OK but what if there is a different response? I think the code could be re-written to use .retrieve() instead probably with .toEntity(Application.class).

I would also double check, is it a leak or is it using connections from the pool?