SageMaker does not support 3rd party dependencies isolation

[lxning]

🚀 The feature

This feature requests to always create model dir under tmp dir, link with the input model dir, and install 3rd party dependency in the tmp to mitigate the issue that SageMaker does not support 3rd party dependencies isolation. Test step:

1. create model artifacts with no-archive format (ie. --archive-format no-archive)

torch-model-archiver --model-name sam --version 1.0 --serialized-file workspace/sam/sam_vit_h_4b8939.pth --handler workspace/sam/custom_handler.py --config-file workspace/sam/model-config.yaml --archive-format no-archive

2. start TS

torchserve --ncs --start --model-store model_store --models sam

Motivation, pitch

SageMaker set /opt/ml/model dir as a model input. This dir is readonly, which means it does not allow TS to install 3rd party dependency in requirements.txt in the model dir.

Alternatives

No response

Additional context

No response

[chauhang]

@lxning Is there anything else needed for this?