Heyang Sun

## Description This is an example of TRL reward modeling, a kind of RLHF, where uses dataset Anthropic/hh-rlhf that on our whitelist. ### 1. Why the change? Enable RM on...

Deepspeed Zero3 QLoRA Fine-tuning

1

## Description Use Deepspeed Zero3 to split and distribute layers of a large model to multiple XPUs and executes QLoRA fine-tuning. ### 1. Why the change? as above ### 2....

Shape Mismatch with Checkpoint for Deepspeed Zero3

1

Tried to load and distribute model to devices in a layerwise way, by using deepspeed zero3 context manager as below: ```python with ds.zero.Init(config_dict_or_path=deepspeed): model = AutoModelForCausalLM.from_pretrained( base_model, config=model_config, torch_dtype=torch.bfloat16, ignore_mismatched_sizes=True,...

## Description ### 1. Why the change? #11167 ### 2. User API changes ### 3. Summary of the change ### 4. How to test? - [ ] N/A - [...