Remove pending acquirePromise once chained responseFuture fails
Closes #1094.
First, I have read #1095, and I understand why it was rejected. I agree having a limit on pending requests is out of scope.
This PR addresses the "memory leak" in a different way.
Despite that the "memory leak" is "accounted for" and "recoverable", there is still a risk of full service meltdown if the service has very high thoughput and the retained memory grows quickly towards HeapOutOfMemory. Although most likely this will be a user error like a bad certificate etc, the possibility of Apple Server having an outage cannot be entirely ruled out. Having the possibility of APNS server going down causing a client service to have HeapOutOfMemory and bring down that whole service is not acceptable.
In the failure path, if the chained responseFuture already fails (e.g. with TimeoutException), the retained acquirePromise effectively becomes useless. Even if it recovers and resolves later, the next chained responseFuture is already resolved as failure, and the listener on the acquirePromise effectively becomes a no-op when attempting to complete a responseFuture that already failed.
Holding the acquirePromise in memory also holds a Listener object, which hold the responseFuture, which holds an Exception, which holds a StackTrace, etc... The total retained memory foot print for just a single acquirePromise is about 1.27KB in a failure case.
With that said, as soon as responseFuture fails, there is no reason for keeping its acquirePromise, therefore we can get rid of it:
- Listen on
responseFuture's failure, and attempt to cancelacquirePromiseif it is cancellable (not resolved yet). - Listen on
acquirePromise's cancel event, and remove it frompendingAcquisitionPromisesto allow GC to collect these unnecessary pending requests immediately.