Support for MultiRecord BatchStrategy in SageMaker Batch Transform

[adelevie]

trafficstars

I'm looking at this example: https://github.com/huggingface/notebooks/blob/main/sagemaker/12_batch_transform_inference/sagemaker-notebook.ipynb

I am trying to get something similar working, but with the MultiRecord BatchStrategy.

Attempt No. 1:

transformer = Transformer(model_name=model_name,
                          instance_count=instance_count,
                          instance_type=instance_type,
                          strategy='MultiRecord',
                          assemble_with='Line',
                          max_payload=45,
                          max_concurrent_transforms=2,
                          sagemaker_session=sagemaker_session,
                          accept='application/jsonlines')

transformer.transform(transform_s3,
                      content_type='application/json',
                      split_type='Line')

This gets the following error:

ClientError: 400

Message:
{
  "code": 400,
  "type": "InternalServerException",  
  "message": "Extra data: line 2 column 1 (char 176)"
}

This error seems consistent with the behavior described here,

For the input in:

input1 

input2

input3

input4

input5

input6

The output would be in below format:

output1,output2,output3

output3,output4,output6

where two distinct JSON documents are passed to json.loads (or equivalent).

Attempt No. 2:

(note content_type='application/jsonlines')

transformer = Transformer(model_name=model_name,
                          instance_count=instance_count,
                          instance_type=instance_type,
                          strategy='MultiRecord',
                          assemble_with='Line',
                          max_payload=45,
                          max_concurrent_transforms=2,
                          sagemaker_session=sagemaker_session,
                          accept='application/jsonlines')

transformer.transform(transform_s3,
                      content_type='application/jsonlines',
                      split_type='Line')

Error:

mms.service.PredictionException: Content type application/jsonlines is not supported by this framework.
Please implement input_fn to to deserialize the request data or an output_fn to
serialize the response. For more information, see the SageMaker Python SDK README. 400

Is there a straightforward way to do this with an existing model trained using HuggingFaceEstimator.fit()?

[danielbis]

+1 on this. @adelevie have you ever figured this out?

[rezadnayeri]

I have similar problem, @adelevie have you ever found a way to fix this?

[dudelstein]

I have a similar issue, is there a solution? Thanks.