sagemaker-python-sdk icon indicating copy to clipboard operation
sagemaker-python-sdk copied to clipboard

Reduce risks of errors due to `CreateProcessingJob` supporting only strings

Open AdrianB-sovo opened this issue 1 year ago • 0 comments

When trying to reference a pipeline parameter of type ParameterInteger when creating a pipeline job, I get the following error: botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the CreatePipeline operation: Cannot assign property reference [Parameters.split_number] to argument of type [String].

I saw that it wasn't supported and that it's not expected to change (https://github.com/aws/sagemaker-python-sdk/issues/2631). Since it could prevent people — like me — from searching what they did wrong when it's just not supported, I think it should be better documented.

I see two ways to improve the documentation:

  • Change the type hints of arguments for the steps.

From:

arguments: Optional[List[Union[str, PipelineVariable]]] = None

to

arguments: Optional[List[Union[str, ParameterString]]] = None
  • Add comments explaining that CreateProcessingJob API only supports strings, so any other type of PipelineVariable should be converted.

AdrianB-sovo avatar Sep 10 '24 00:09 AdrianB-sovo