JQFeng

Problem in building Alpa-modified Jaxlib.

5

**Please describe the bug** **Please describe the expected behavior** **System information and environment** - OS Platform and Distribution (e.g., Linux Ubuntu 16.04, docker): - Python version: 3.9 - CUDA version:11.3...

About the parameters in example.json

1

I want to know if the parameters in the original `example.json` is reasonable. Or could u give me some recommended parameters? Thanks!

unreadable generation results

1

# Problems when use 2 A800 GPU serving LLaMA-3-8B IIRC, no other change besides this. It's ok with one GPU. BTW what's the difference bwtween `actual_gpu_num` and `gpu_num`