FuelSDK-Node icon indicating copy to clipboard operation
FuelSDK-Node copied to clipboard

Connection exceptions after upgrade

Open WithSecureSamiTikka opened this issue 8 years ago • 1 comments

We run AWS Lambda function which uses FuelSDK to update a Data Extension. We recently upgraded the SDK from 1.1.8 to 2.1.0. After the upgrade the lambda function started crashing with following errors

START RequestId: 7adcc610-33b7-4c1d-9ebe-38ad8957525e Version: 37
2016-12-03T04:56:14.799Z	4722357d-fda6-49d0-87df-28538e923b27	Error: connect ECONNRESET 198.245.89.89:443
at Object.exports._errnoException (util.js:870:11)
at exports._exceptionWithHostPort (util.js:893:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1062:14)
END RequestId: 7adcc610-33b7-4c1d-9ebe-38ad8957525e
REPORT RequestId: 7adcc610-33b7-4c1d-9ebe-38ad8957525e	Duration: 35.29 ms	Billed Duration: 100 ms Memory Size: 256 MB	Max Memory Used: 49 MB	
Process exited before completing request

Also

START RequestId: 30200f4c-20df-49e7-9876-b18635d7dd94 Version: 37
2016-12-03T04:56:59.816Z	396a31ac-1671-46ad-818f-61f9b8597cc3	Error: socket hang up
at createHangUpError (_http_client.js:200:15)
at TLSSocket.socketOnEnd (_http_client.js:285:23)
at emitNone (events.js:72:20)
at TLSSocket.emit (events.js:166:7)
at endReadableNT (_stream_readable.js:905:12)
at nextTickCallbackWith2Args (node.js:437:9)
at process._tickDomainCallback (node.js:392:17)
END RequestId: 30200f4c-20df-49e7-9876-b18635d7dd94
REPORT RequestId: 30200f4c-20df-49e7-9876-b18635d7dd94	Duration: 34.86 ms	Billed Duration: 100 ms Memory Size: 256 MB	Max Memory Used: 52 MB	
Process exited before completing request

These errors seem to mostly happen during slow traffic hours and before the error there has been 1-2 minutes of idle time. My hypothesis is the Exact Target server side has closed the socket and because that is not handled, it causes an exception which kills the process.

Has something changed in the SDK because this problem started happening after the SDK upgrade?

According to the logs, the exception occurs before any of our code runs in the lambda function, making it impossible to put a try+catch around it. Would it be possible to either:

  • handle the error in the SDK or
  • emit the error as an event from the SDK to allow the application to handle the error

WithSecureSamiTikka avatar Dec 03 '16 08:12 WithSecureSamiTikka

It appears that the errors are caused by an extra call to the API performed in the background whenever the SOAP api is used: https://github.com/salesforce-marketingcloud/FuelSDK-Node/blob/master/lib/ET_Client.js#L79-L86

That request happens outside all normal control flow and no one is handling the response. If that request fails for one reason or another (in this case the Lambda function container was frozen for some minutes while the request was still in-flight), the error will be converted into an uncaught exception and causing the process to crash.

Git history reveals that this code was added in PR #6. @amagar088, what was the reason for adding this extra request to the library? Could this phone-home functionality be implemented in some other way as part of the normal processing flow so that errors from that can be handled by the application?

sjakthol avatar Jun 02 '17 05:06 sjakthol