worker-vllm icon indicating copy to clipboard operation
worker-vllm copied to clipboard

Published 20 hours ago • verified publisher icon runpod-workers

Building Docker with model built in

Open KDercksen opened this issue 2 months ago • 3 comments

Hi there,

The current version of the download_model.py script does not work due to the empty TENSORIZE_MODEL env check on line 50.

Once that is fixed, the weight_utils file in the vllm-base image does not exist - it seems there is some version mismatch going on with the vllm submodule and the new 1.0.0preview image.

Could you take a look?

KDercksen avatar May 15 '24 07:05 KDercksen