Chen Cui

Sequence Parallel for LoRA

3

# What does this PR do ? Allow sequence parallel for lora training **Collection**: NLP # Changelog - Set sequence_parallel flags to False in LoRA layers to allow gathering output...

Multi-GPU QLoRA?

6

Hi, first of all thanks for the great tutorials on lora and qlora! I was able to follow them very easily. I was wondering if multi-gpu QLoRA is supported? I...

# What does this PR do ? - Remove deprecated files from #8201 - Remove temporary changes due to #8590 **Collection**: [Note which collection this PR will affect] # Changelog...

# What does this PR do ? - Fix circular import of `nemo.collections.multimodal.models.text_to_image.stable_diffusion.ldm.autoencoder.AutoencoderKL` - Rename NeMo-Megatron-Launcher to NeMo-Framework-Launcher **Collection**: [Note which collection this PR will affect] # Changelog - Add...

# What does this PR do ? - Pull out core sequence packing logic to nemo.utils so they can be used elsewhere (e.g. NeMo-Aligner) - Change to loading only tokenizer...

Add offline quantization script for QLoRA deployment

1

# What does this PR do ? Add an offline quantization script to convert a base model to NF4 (but cast back to BF16 for storage) Enables deployment of QLoRA...

[NeMo-UX] Add PEFT

6

# What does this PR do ? Initial PR for PEFT in nemo 2.0 **Collection**: [Note which collection this PR will affect] # Changelog - Add specific line by line...

# What does this PR do ? Support QLoRA for more models **Collection**: [Note which collection this PR will affect] # Changelog - Add specific line by line info of...

# What does this PR do ? Add finetuning recipes for Baichuan, ChatGLM, Gemma **Collection**: [Note which collection this PR will affect] # Changelog - Add specific line by line...

[Cherrypick #10445 and #10898] Packed Sequence [NeMo 2]

1

# What does this PR do ? Add a one line overview of what this PR aims to accomplish. **Collection**: [Note which collection this PR will affect] # Changelog -...