solitude-alive

This is the lora single device test: This is the qlora single device test:

LoRA single device: ``` INFO:torchtune.utils.logging:Memory Stats after model init: {'peak_memory_active': 5.547595264, 'peak_memory_alloc': 5.547595264, 'peak_memory_reserved': 5.559549952} Step 100 | peak_memory_active:14.325987328 peak_memory_alloc:14.325987328 peak_memory_reserved:24.664604672 ``` QLoRA single device: ``` INFO:torchtune.utils.logging:Memory Stats after model...