Jamil Brian Palma Salazar

thanks @nvluxiaoz At the end I downloaded version [llama hf version](https://huggingface.co/meta-llama/Llama-2-7b-hf) from Hugging Face, which solved the initial problem with the config.json file. I'm not sure if it's because I'm...

root@b45ee7ad85b4:/TensorRT-LLM/examples/llama# ls Llama-2-7b-hf convert_checkpoint.py summarize_long.py tllm_checkpoint_2gpu_tp2 README.md requirements.txt tmp root@b45ee7ad85b4:/TensorRT-LLM/examples/llama# cd Llama-2-7b-hf/ root@b45ee7ad85b4:/TensorRT-LLM/examples/llama/Llama-2-7b-hf# ls LICENSE.txt config.json model.safetensors.index.json special_tokens_map.json README.md generation_config.json pytorch_model-00001-of-00002.bin tokenizer.json Responsible-Use-Guide.pdf model-00001-of-00002.safetensors pytorch_model-00002-of-00002.bin tokenizer.model USE_POLICY.md model-00002-of-00002.safetensors pytorch_model.bin.index.json tokenizer_config.json...

now I have this issue > `root@c6fc756c94d5:/TensorRT-LLM/examples/llama# mpirun -n 8 --allow-run-as-root python3 ../run.py --max_output_len=50 --engine_dir ./tmp/llama/7B/trt_engines/fp16/8-gpu/ --input_text "To tell a story" [TensorRT-LLM][INFO] Engine version 0.9.0.dev2024020600 found in the config file,...

Thanks, you're right; I'll rephrase the question to make it clearer in summary, I'm using the Virtual Machine Scale Set (VMSS) because these are created when deploying EchoBot the issue...

@InDieTasten thank you for the response. I also believe that here it should be mentioned that I tried to deploy EchoBot again with the original script and also by changing...

Hello, I got the same error, could anyone have any progress?