ptschandl

Constrained system/cpu RAM prohibits loading even with enough GPU Memory

2

### System Info The issue occurred with `OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5` on a machine with 32GB RAM and a single RTX 6000 Ada (48GB), where the shard loading aborts, but loading with raw...