analytics-next icon indicating copy to clipboard operation
analytics-next copied to clipboard

Setting a low `flushInterval` effectively breaks `closeAndFlush()`

Open daguej opened this issue 9 months ago • 2 comments

We're using @segment/analytics-node in a serverless environment that is very strict about ending execution after the main entry point's promise resolves. The hosting node process is killed immediately after the promise resolves, so if there was anything still happening in the background, it is lost.

We ~followed~ misread the guidance found in the readme and accidentally set flushInterval to 1. We found some events were being dropped, particularly if fired shortly before the serverless function finished and resolved.

So we then added await analytics.closeAndFlush() to the end of our function to ensure events were being flushed before exiting.

To our surprise, this did not help; events were still being lost.

I believe analytics-node.ts#L47 is to blame. The library sets the default timeout for closeAndFlush() to flushInterval * 1.25, meaning our call to closeAndFlush() was quietly resolving after 1.25ms, which is definitely not enough time to establish a TLS connection to Segment's servers and send any events.

Thus, our function resolved its promise before analytics events could hit the wire, and the process was killed before the events could be sent.

This timeout behavior was not documented anywhere, so it was very surprising to find out that the flush call wasn't working; we had no idea we'd need to set a sane timeout ourselves (await analytics.closeAndFlush({ timeout: 10000 })).

  • The default should probably be something like this._closeAndFlushDefaultTimeout = Math.max(5000, flushInterval * 1.25) so that a low interval doesn't break flushing, and the docs should better describe this behavior.
  • This is totally my opinion, but the readme's advice to set ~flushInterval~ flushAt to 1 and await every track call seems…not great. Isn't it preferable to keep the default interval, use non-blocking track calls, and then closeAndFlush() before returning from your serverless function?

daguej avatar May 22 '24 17:05 daguej