tensorrtllm_backend icon indicating copy to clipboard operation
tensorrtllm_backend copied to clipboard

Example of LoRa weights

Open TheCodeWrangler opened this issue 10 months ago • 2 comments

I would like to send Lora weights through to a compiled tensor rt llm model but am unsure how to load the .bin weights and pass them to Triton. An example of using them and passing in weights would be very helpful

TheCodeWrangler avatar Apr 09 '24 21:04 TheCodeWrangler