text-generation-inference icon indicating copy to clipboard operation
text-generation-inference copied to clipboard
verified publisher icon huggingface

MODEL_ID propagation fix

Open seungrokj opened this issue 1 year ago • 2 comments

Hi,

This is to fix the bug here: File "/opt/conda/lib/python3.10/site-packages/text_generation_server/models/flash_causal_lm.py", line 1160, in warmup f"tunableop_{MODEL_ID.replace('/', '-')}_tp{self.world_size}_rank{self.rank}.csv",

MODEL_ID is not properly propaged.

Please review this @OlivierDehaene @Narsil

Regards, Seungrok

seungrokj avatar Jul 24 '24 03:07 seungrokj