Mustapha Unubi Momoh

Resolved. Thanks @BLSharda @anujj . Will close the issue now.

@shishirgoyal85 @anujj Installing tensorrt_llm version 0.5 i.e. `pip install tensorrt_llm==0.5 --extra-index-url https://pypi.nvidia.com/ --extra-index-url https://download.pytorch.org/whl/cu121` results in the error: ``` ERROR: No matching distribution found for torch==2.1.0.dev20230828+cu121 ``` Here is the...

Run the setup script provided. The script you are looking for is under the `windows` folder. You can skip `Python` and `MPI` if they are already correctly installed. Make sure...

> Please try changing an environment with more RAM. Thanks! Thanks @Shixiaowei02 My system specs are: Win 11: 64G RAM, and 24G VRAM (4090). Isn't this enough?

Thanks @Shixiaowei02 I was able to build with docker on windows. Will close the issue now.

> Hi @MustaphaU, thanks for reaching out. If you limit the script to only be initializing a client (no actual operations), do you still have this behavior? In other words,...

@RyanFitzSimmonsAK Please see the attached below from cloudwatch logs: Also, see the relevant part of the inference script: You would observe from the log that execution failed at the point...

@RyanFitzSimmonsAK Thanks. Not following an example notebook or deploying in a VPC. I have created a repo with instructions to reproduce the issue here: https://github.com/MustaphaU/rerror

> I was facing the same issue when trying to build a sagemaker serving tenserflow image. adding monkey patch to python_service.py at the very top helped me. > > ```...

> @deepblue-phoenix Would you mind sharing your workaround for this issue? > > Does anyone have a solution or a timeline on this? You could try the [suggestions](https://github.com/boto/boto3/issues/4061#issuecomment-2130118141) by @shresthapradip...