Shilong Lei issues

Repositories
Issues
Comments

Results 1 issues of


                                            Shilong Lei

[BUG] Inference a large model using NVMe offload. AssertionError: More elements 524709888 than buffer size 100,000,000

**Describe the bug** I'm trying to run inference of a 54 billion model (facebook/nllb-moe-54b) using NVMe offload on my laptop with a RTX 3060 (6GB GPU memory). But I get...

bug

inference