[client-lambda] Cannot destructure property 'response' of '(intermediate value)' as it is undefined

[jlowcs]

Checkboxes for prior research

• [x] I've gone through Developer Guide and API reference
• [x] I've checked AWS Forums and StackOverflow.
• [x] I've searched for previous similar issues and didn't find any solution.

Describe the bug

Cannot destructure property 'response' of '(intermediate value)' as it is undefined is reported to Sentry pretty regularly when using @aws-sdk/client-lambda.

Regression Issue

• [ ] Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/[email protected]

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

node v22.11.0

Reproduction Steps

Cannot reproduce easily.

We're calling lambda.invoke the following way:

import { Lambda } from '@aws-sdk/client-lambda';
import { NodeHttpHandler } from '@smithy/node-http-handler';

const lambda = new Lambda({
  region: 'us-west-2',
  apiVersion: '2015-03-31',
  requestHandler: new NodeHttpHandler({
    connectionTimeout: 1000,
    socketTimeout: 1000,
  }),
});

// ...

const response = await lambda.invoke({
  FunctionName: longlinksStoreLambdaFunctionName,
  InvocationType: 'RequestResponse',
  LogType: 'None',
  Payload: JSON.stringify({
    body: JSON.stringify({
      url,
      hashLength,
    }),
  }),
});

I just noticed that our apiVersion value is quite old btw.

Observed Behavior

The following error is being reported to Sentry:

Note that this doesn't happen every time, so it "feels" like a race condition.

Expected Behavior

No error

Possible Solution

No response

Additional Information/Context

See also https://github.com/smithy-lang/smithy-typescript/issues/1558

[kuhe]

Information for SDK maintainers:

The deserializerMiddleware is the last middleware in most stacks, including the default stack of the Lambda client.

To confirm, run the following code to make the client log the middleware stack:

const client = new Lambda(...);
client.middlewareStack.identifyOnResolve(true);

await client.invoke(...);

Output will resemble:

[
  'loggerMiddleware - initialize',
  'httpAuthSchemeMiddleware - serialize',
  'endpointV2Middleware - serialize',
  'serializerMiddleware - serialize',
  'contentLengthMiddleware - build',
  'getUserAgentMiddleware - build',
  'hostHeaderMiddleware - build',
  'recursionDetectionMiddleware - build',
  'retryMiddleware - finalizeRequest',
  'httpSigningMiddleware (a.k.a. apiKeyMiddleware,tokenMiddleware,awsAuthMiddleware) - finalizeRequest',
  'deserializerMiddleware - deserialize'
]

Once the deserializerMiddleware is confirmed to be the final middleware, the only downstream invocation is the requestHandler.handle() async method. In Node.js this is the NodeHttpHandler.

Try to find a condition where the handle method returns an undefined value without throwing, since that seems like the only way to cause this issue.

[jlowcs]

My output:

[
  'loggerMiddleware - initialize',
  'httpAuthSchemeMiddleware - serialize',
  'endpointV2Middleware - serialize',
  'serializerMiddleware - serialize',
  'contentLengthMiddleware - build',
  'getUserAgentMiddleware - build',
  'hostHeaderMiddleware - build',
  'recursionDetectionMiddleware - build',
  'retryMiddleware - finalizeRequest',
  'httpSigningMiddleware (a.k.a. apiKeyMiddleware,tokenMiddleware,awsAuthMiddleware) - finalizeRequest',
  'deserializerMiddleware - deserialize'
]

Try to find a condition where the handle method returns an undefined value without throwing, since that seems like the only way to cause this issue.

Where could I search for that since NodeHttpHandler is itself coming from a library?

[jlowcs]

Looking at NodeHttpHandler's code, it seems to always be returning a promise:

[aBurmeseDev]

Hi @jlowcs - appreciate your patience on this.

Assuming this's still an issue, I'd need to ask for more details to be able to reproduce it on my end. Are you using your own custom middleware and how intermittent does this occur? Could you maybe provide some minimal steps for us to try to reproduce this on our end?

[jlowcs]

@aBurmeseDev it is still happening. I'm not using any custom middleware. The way I use it couldn't be more simple:

import { Lambda } from '@aws-sdk/client-lambda';
import { NodeHttpHandler } from '@smithy/node-http-handler';

const lambda = new Lambda({
  region: 'us-west-2',
  apiVersion: '2015-03-31',
  requestHandler: new NodeHttpHandler({
    connectionTimeout: 1000,
    socketTimeout: 1000,
  }),
});

// ...

const response = await lambda.invoke({ // it throws here
  FunctionName: longlinksStoreLambdaFunctionName,
  InvocationType: 'RequestResponse',
  LogType: 'None',
  Payload: JSON.stringify({
    body: JSON.stringify({
      url,
      hashLength,
    }),
  }),
});

[aBurmeseDev]

I can't reproduce it on my end with similar to your code (see below). Are you able to share the rest of the code you have right before calling invoke?

[jlowcs]

It's pretty random so I'm not surprised you can't reproduce it. We're seeing a few failures every thousands of calls. Maybe you'd see it by running a benchmark.

My guess is that there is something akin to a network issue that somehow results in next() returning something that's not a promise.

Edit: actually, per the error, it is a promise, but the promise holds undefined.

I guess I could try to patch the code that throws to add some kind of logging, or throw an error with more info.

[jlowcs]

There's not much else from our code that I can share. What I shared is basically everything that relates to the lambda call.

[aBurmeseDev]

actually, per the error, it is a promise, but the promise holds undefined. I guess I could try to patch the code that throws to add some kind of logging, or throw an error with more info.

Please share any additional logs with more info or how we can reproduce it on our ends. We'll be better able to help and identify the issue if you can demonstrate that it happens consistently and we can verify it ourselves.

[jlowcs]

The issue seems related to the de_InvokeCommand deserializer, for which the next() function sometimes resolves to undefined. Any suggest as to where I should add some additional logging?

[mscharley]

We're also running into this after updating the AWS SDK recently. We did quite a large jump from 3.750.0 -> 3.917.0, however if @jlowcs was seeing it on 3.787.0 then that narrows the regression window quite a bit.