Runpeng Geng

icon indicating a website link sleeepeer.github.io icon indicating an email [email protected]

icon location State College, United States icon location PhD student @PennState.　　　　Bachelor of CS, WHU.　　　　　Researcher in Trustworthy AI.

Results 1 issues of Runpeng Geng

Better device_map and max_memory for loading vicuna model

Hi! Thank you for this wonderful repo. When I was trying to load vicuna model with limited VRAM across different GPUs. I discovered that your "max_memory" part would cause the...