[SageMakerRuntime] got error when invoking async inference: Value at 'body' failed to satisfy constraint: Member must have length less than or equal to 0"

[tan-t]

Describe the bug

I got the error below when call SagemakerRuntime#invokeEndpointAsync from lambda Node16.x runtime even though it doesn't look it's violating any constraints.

Expected Behavior

to Start an async inference without raising error

Current Behavior

the error:

{     "errorType": "ValidationError",     "errorMessage": "1 validation error detected: Value at 'body' failed to satisfy constraint: Member must have length less than or equal to 0",     "code": "ValidationError",     "message": "1 validation error detected: Value at 'body' failed to satisfy constraint: Member must have length less than or equal to 0",     "time": "2022-09-10T11:40:22.985Z",     "requestId": "86958177-7161-4f14-bd24-b1cf52089603",     "statusCode": 400,     "retryable": false,     "retryDelay": 93.45195287738557,     "stack": [         "ValidationError: 1 validation error detected: Value at 'body' failed to satisfy constraint: Member must have length less than or equal to 0",         "    at Object.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/json.js:52:27)",         "    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/rest_json.js:49:8)",         "    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)",         "    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)",         "    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:686:14)",         "    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)",         "    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)",         "    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10",         "    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)",         "    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:688:12)"     ] } | 2022-09-10T11:40:23.015Z b837bfb5-5c85-43f6-aebc-5f418c21b597 ERROR Invoke Error {"errorType":"ValidationError","errorMessage":"1 validation error detected: Value at 'body' failed to satisfy constraint: Member must have length less than or equal to 0","code":"ValidationError","message":"1 validation error detected: Value at 'body' failed to satisfy constraint: Member must have length less than or equal to 0","time":"2022-09-10T11:40:22.985Z","requestId":"86958177-7161-4f14-bd24-b1cf52089603","statusCode":400,"retryable":false,"retryDelay":93.45195287738557,"stack":["ValidationError: 1 validation error detected: Value at 'body' failed to satisfy constraint: Member must have length less than or equal to 0"," at Object.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/json.js:52:27)"," at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/rest_json.js:49:8)"," at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)"," at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)"," at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:686:14)"," at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)"," at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)"," at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10"," at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)"," at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:688:12)"]}

the main error is Value at 'body' failed to satisfy constraint: Member must have length less than or equal to 0

it felt strange, as

• I have passed nothing named 'body' to the SDK ,
• is says 'Member must have length less than or equal to 0', not 'more' but 'less'. it looks like I'm putting something extra to the SDK, but I can't found which.

Reproduction Steps

the lambda has no layers imported, and has full access privillege to both of S3 and Sagemaker.
aws-sdk's version was 2.1083.0.

the code:

const AWS = require("aws-sdk");

exports.handler = async (event) => {
  const input = event.queryStringParameters;
  const key = input.inferenceId;

  // you may skip this since it is environment-specific and unnecessary for reproduction ===
  await new AWS.S3()
    .putObject({
      Bucket: "marble-input",
      Key: `${key}.json`,
      Body: JSON.stringify(input),
    })
    .promise();
  // =========================================================

  try {
    // maybe you should change the args for your environment =============
    await new AWS.SageMakerRuntime()
      .invokeEndpointAsync({
        InputLocation: `s3://marble-input/${key}.json`,
        EndpointName: "diffusers-async",
      })
      .promise();
    // ==================================================
  } catch (e) {
    console.log(e);
    throw e;
  }

  return {
    statusCode: 200,
    body: JSON.stringify({ success: true }),
    headers: {
      "Content-Type": "application/json",
    },
  };
};

the test-event:

{
  "body": "eyJ0ZXN0IjoiYm9keSJ9",
  "resource": "/{proxy+}",
  "path": "/path/to/resource",
  "httpMethod": "POST",
  "isBase64Encoded": true,
  "queryStringParameters": {
    "prompt": "cat soldier",
    "inferenceId": "1"
  },
  "multiValueQueryStringParameters": {
    "foo": [
      "bar"
    ]
  },
  "pathParameters": {
    "proxy": "/path/to/resource"
  },
  "stageVariables": {
    "baz": "qux"
  },
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, sdch",
    "Accept-Language": "en-US,en;q=0.8",
    "Cache-Control": "max-age=0",
    "CloudFront-Forwarded-Proto": "https",
    "CloudFront-Is-Desktop-Viewer": "true",
    "CloudFront-Is-Mobile-Viewer": "false",
    "CloudFront-Is-SmartTV-Viewer": "false",
    "CloudFront-Is-Tablet-Viewer": "false",
    "CloudFront-Viewer-Country": "US",
    "Host": "1234567890.execute-api.us-east-1.amazonaws.com",
    "Upgrade-Insecure-Requests": "1",
    "User-Agent": "Custom User Agent String",
    "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)",
    "X-Amz-Cf-Id": "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==",
    "X-Forwarded-For": "127.0.0.1, 127.0.0.2",
    "X-Forwarded-Port": "443",
    "X-Forwarded-Proto": "https"
  },
  "multiValueHeaders": {
    "Accept": [
      "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"
    ],
    "Accept-Encoding": [
      "gzip, deflate, sdch"
    ],
    "Accept-Language": [
      "en-US,en;q=0.8"
    ],
    "Cache-Control": [
      "max-age=0"
    ],
    "CloudFront-Forwarded-Proto": [
      "https"
    ],
    "CloudFront-Is-Desktop-Viewer": [
      "true"
    ],
    "CloudFront-Is-Mobile-Viewer": [
      "false"
    ],
    "CloudFront-Is-SmartTV-Viewer": [
      "false"
    ],
    "CloudFront-Is-Tablet-Viewer": [
      "false"
    ],
    "CloudFront-Viewer-Country": [
      "US"
    ],
    "Host": [
      "0123456789.execute-api.us-east-1.amazonaws.com"
    ],
    "Upgrade-Insecure-Requests": [
      "1"
    ],
    "User-Agent": [
      "Custom User Agent String"
    ],
    "Via": [
      "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)"
    ],
    "X-Amz-Cf-Id": [
      "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA=="
    ],
    "X-Forwarded-For": [
      "127.0.0.1, 127.0.0.2"
    ],
    "X-Forwarded-Port": [
      "443"
    ],
    "X-Forwarded-Proto": [
      "https"
    ]
  },
  "requestContext": {
    "accountId": "123456789012",
    "resourceId": "123456",
    "stage": "prod",
    "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef",
    "requestTime": "09/Apr/2015:12:34:56 +0000",
    "requestTimeEpoch": 1428582896000,
    "identity": {
      "cognitoIdentityPoolId": null,
      "accountId": null,
      "cognitoIdentityId": null,
      "caller": null,
      "accessKey": null,
      "sourceIp": "127.0.0.1",
      "cognitoAuthenticationType": null,
      "cognitoAuthenticationProvider": null,
      "userArn": null,
      "userAgent": "Custom User Agent String",
      "user": null
    },
    "path": "/prod/path/to/resource",
    "resourcePath": "/{proxy+}",
    "httpMethod": "POST",
    "apiId": "1234567890",
    "protocol": "HTTP/1.1"
  }
}

Possible Solution

the Sagemaker Runtime API (and the inference endpoint) itself seems to be working fine though ,as I tested it on sagemaker studio notebook with boto3 client.
so I have looked into this repository itself, but I found nothing suspicious.
this json seems to represent the spec of the api and this SDK seems to just load the spec-representating-jsons and behave just as written. https://github.com/aws/aws-sdk-js/blob/master/apis/runtime.sagemaker-2017-05-13.normal.json
I have nothing to suggest but I doubt it is not calling API properly or something.

Additional Information/Context

No response

SDK version used

2.1083.0

Environment details (OS name and version, etc.)

AWS Lambda node 16.x runtime. the lambda has no layers imported, and has full access privillege to both of S3 and Sagemaker.

[tan-t]

may it be the root cause?

• this SDK defines SagemakerRuntime.invokeEndpointAsync operation as using RestJson protocol. https://github.com/aws/aws-sdk-js/blob/master/apis/runtime.sagemaker-2017-05-13.normal.json#L7
• this SDK calls JsonBuilder.build() when building request body, with req.params as argument. https://github.com/aws/aws-sdk-js/blob/master/lib/protocol/rest_json.js#L27
• the JsonBuilder builds it to {} instead of an empty string https://github.com/aws/aws-sdk-js/blob/master/test/json/builder.spec.js#L61
• the InvokeEndpointAsync API needs no request body.
• therefore the forementioned error occurs

[tan-t]

yes this fails

(function () {
  const helpers = require('./helpers');
  const { AWS,  spyOn } = helpers;

  return describe('SagemakerRuntime.invokeEndpointAsync', function () {
    it('should call api with an empty body', async () => {
      const httpClient = AWS.HttpClient.getInstance();
        spyOn(httpClient, 'handleRequest')
        .andCallFake(function (httpReq, httpOp, cb, errCb) {
          expect(httpReq.body).to.equal('');
          helpers.mockHttpSuccessfulResponse(
            200,
            {},
            JSON.stringify({ success: true }),
            cb
          );
        });
      await new AWS.SageMakerRuntime()
        .invokeEndpointAsync({
          InputLocation: 's3://mock',
          EndpointName: 'mock',
        })
        .promise();
    });
  });
}.call(this));

     NetworkingError: expected '{}' to equal ''
      + expected - actual

      -{}

[tan-t]

I editted the SDK and the issue has been no longer reproduced.
the PR is #4204 .

I tried this:

(in node REPL on master branch)

const AWS = require('./index.js');
const client = new AWS.SageMakerRuntime({region: 'ap-northeast-1', accessKeyId: MY_ACCESS_KEY, secretAccessKey: MY_SECRET_ACCESS_KEY});
await client.invokeEndpointAsync({InputLocation: 's3://marble-result/input.json', EndpointName: 'diffusers-async'}).promise();

then got the same error:

Uncaught:
Error [ValidationError]: 1 validation error detected: Value at 'body' failed to satisfy constraint: Member must have length less than or equal to 0
    at Object.extractError (/home/tan_t/workspace/aws-sdk-js/lib/protocol/json.js:52:27)
    at Request.extractError (/home/tan_t/workspace/aws-sdk-js/lib/protocol/rest_json.js:49:8)
    at Request.callListeners (/home/tan_t/workspace/aws-sdk-js/lib/sequential_executor.js:106:20)
    at Request.emit (/home/tan_t/workspace/aws-sdk-js/lib/sequential_executor.js:78:10)
    at Request.emit (/home/tan_t/workspace/aws-sdk-js/lib/request.js:686:14)
    at Request.transition (/home/tan_t/workspace/aws-sdk-js/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/home/tan_t/workspace/aws-sdk-js/lib/state_machine.js:14:12)
    at /home/tan_t/workspace/aws-sdk-js/lib/state_machine.js:26:10
    at Request.<anonymous> (/home/tan_t/workspace/aws-sdk-js/lib/request.js:38:9) {
  code: 'ValidationError',
  time: 2022-09-10T14:42:50.298Z,
  requestId: 'bea52e19-8a7b-42ee-8921-cb95623f993f',
  statusCode: 400,
  retryable: false,
  retryDelay: 80.39644862544108
}

(in node REPL on fix/4203 branch)

const AWS = require('./index.js');
const client = new AWS.SageMakerRuntime({region: 'ap-northeast-1', accessKeyId: MY_ACCESS_KEY, secretAccessKey: MY_SECRET_ACCESS_KEY});
await client.invokeEndpointAsync({InputLocation: 's3://marble-result/input.json', EndpointName: 'diffusers-async'}).promise();

then got the expected response:

{
  OutputLocation: 's3://marble-result/ad41e460-91f1-4f92-b492-2059a1de9ddc.out',
  InferenceId: '0e4b1ee5-5b9d-4571-9eeb-694995be950b'
}

[ajredniwja]

Hey @tan-t thanks for opening this issue and doing all the extensive research too, I was able to reproduce the issue too, I saw you have created the PR too, I would need to discuss a little with the team and will have it reviewed soon. Thanks

[tan-t]

@ajredniwja thank you for the reply!
I'm glad to hear that you have reproduced the issue.
I'm looking forward getting my PR reviewed, if any changes needed please let me know!

[cmdbbang]

same issue here when invoke async endpoint.

const params = {
    ContentType: "application/json",
    EndpointName: "endpoitname",
    InputLocation: "s3key",
  };
await sageMakerRuntime.invokeEndpointAsync(params).promise();

aws-sdk's version is 2.1083.0.

Error [ValidationError]: 1 validation error detected: Value at 'body' failed to satisfy constraint: Member must have length less than or equal to 0

[tan-t]

I had created a simple PR https://github.com/aws/aws-sdk-js/pull/4204/files several months ago.
it fixes the issue mentioned above and has its unit test case in it.
it is not merged yet, but you can use its code from wherever you like.

const AWS = require('aws');
AWS.util.update(AWS.SageMakerRuntime.prototype, {
  /**
   * @api private
   */
  setupRequestListeners: function setupRequestListeners(request) {
    if (request.operation === 'invokeEndpointAsync') {
      request.addListener('build', this.emptyBody);
    }
  },

  /**
   * Empty request body for async inference
   * @api private
   */
  emptyBody: function emptyBody(request) {
    request.httpRequest.body = '';
  },
});

//  then write your own code...

hope it helps!

[cmdbbang]

Hey tan-t, yes it works like a charm. Thanks!

[aBurmeseDev]

If anyone runs into this, please use the workaround mentioned above and kudos to @tan-t for sharing the workaround with others! Closing the issue.