Vincenzo di Cicco

Unable to `finetune/lora.py` with `DDP`

3

Hello, I'm trying different distributed training strategies changing Fabric's strategy argument to different values (as listed [here](https://lightning.ai/docs/fabric/stable/api/fabric_args.html#strategy)). To sanity check, I'm verifying the training is able to overfit a small...

Question about finetuning on large datasets (1-50M samples)

2

Hello, I'm using llama-recipes to experiment finetuning of Llama2 on different dataset sizes (spanning 1M to 50M samples and different configuration). I've noticed a couple of things that make me...

Mixtral with TP hangs indefinitely if another process uses the same GPU with ONNX

3

### System Info - CPU architecture: x86_64 - GPU name: NVIDIA A40, 46GB - TensorRT-LLM: v0.9.0 - Os: Ubuntu 20.04 - Nvidia Driver: 535.54.03, Cuda: 12.2 ### Who can help?...

`-1` token id with Mixtral FP8 and tensorrt_llm 0.11.0

2

- CPU architecture: x86_64 - GPU: NVIDIA H100 - Libraries - TensorRT-LLM: v0.11.0 - TensorRT: 10.1.0 - Modelopt: 0.13.1 - CUDA: 12.3 - NVIDIA driver version: 535.129.03 Hello, I'm experiencing...