Lucius

icon indicating an email [email protected]

icon company Tsinghua University icon location Beijing, China icon location Postgraduate student major in Software Engineering

Results 2 issues of Lucius

Fix args for `decode_batch` in Blip2

**Issue**: In `examples/blip2/run.py`, there was an issue with the way arguments were passed to the `decode_batch` method. This led to a TypeError, with the specific error message: `GenerationSession.decode_batch() takes 3...

Unexpected Memory Usage and Latency with PP

4 comment

When running the `examples/llama/pippy_llama.py` script on two A800 GPUs, each rank is observed to consume the full model_size in memory, rather than sharing the weights across both GPUs. Additionally, the...