hpcaitech
[DOC]: How to do the supervised fine-tuning (SFT) (Stage 1 of chatGPT) ๏ผ
๐ The doc issue
We only see stage 2 and 3 in the chatpgt/example/readme file. Will you please show us how to finetune the gpt/blooom/opt model with an example dataset ?
Bot detected the issue body's language is not English, translate it automatically. ๐ฏ๐ญ๐ป๐งโ๐คโ๐ง๐ซ๐ง๐ฟโ๐คโ๐ง๐ป๐ฉ๐พโ๐คโ๐จ๐ฟ๐ฌ๐ฟ
I really don't know how to do this first step
Simple or not, the absence of the first stage makes the entire demo incomplete. The finetuing for different models maybe very different. Many beginners feel confused about it. If the authors hasnot finished stage 1 themselves, how can they say easy? If the first step has been done, is there any difficulty in sharing relevant resources?
Iโve spent some time drawing a diagram to illustrate the calling process, share with you all
Here is the code to fine-tune the Bloom model, petals
from petals import DistributedBloomForCausalLM
model = DistributedBloomForCausalLM.from_pretrained("bigscience/bloom-petals", tuning_mode="ptune", pre_seq_len=16)
# Embeddings & prompts are on your device, BLOOM blocks are distributed across the Internet
inputs = tokenizer("A cat sat", return_tensors="pt")["input_ids"]
outputs = model.generate(inputs, max_new_tokens=5)
print(tokenizer.decode(outputs[0])) # A cat sat on a mat...
# Fine-tuning (updates only prompts or adapters hosted locally)
optimizer = torch.optim.AdamW(model.parameters())
for input_ids, labels in data_loader:
outputs = model.forward(input_ids)
loss = cross_entropy(outputs.logits, labels)
optimizer.zero_grad()
loss.backward()
optimizer.step()
Bot detected the issue body's language is not English, translate it automatically. ๐ฏ๐ญ๐ป๐งโ๐คโ๐ง๐ซ๐ง๐ฟโ๐คโ๐ง๐ป๐ฉ๐พโ๐คโ๐จ๐ฟ๐ฌ๐ฟ
Iโve spent some time drawing a diagram to illustrate the calling process, share with you all
Iโve spent some time drawing a diagram to illustrate the calling process, share with you all
Here is the code to fine-tune the Bloom model, petals
from petals import DistributedBloomForCausalLM model = DistributedBloomForCausalLM.from_pretrained("bigscience/bloom-petals", tuning_mode="ptune", pre_seq_len=16) # Embeddings & prompts are on your device, BLOOM blocks are distributed across the Internet inputs = tokenizer("A cat sat", return_tensors="pt")["input_ids"] outputs = model.generate(inputs, max_new_tokens=5) print(tokenizer.decode(outputs[0])) # A cat sat on a mat... # Fine-tuning (updates only prompts or adapters hosted locally) optimizer = torch.optim.AdamW(model.parameters()) for input_ids, labels in data_loader: outputs = model.forward(input_ids) loss = cross_entropy(outputs.logits, labels) optimizer.zero_grad() loss.backward() optimizer.step()
Awesome! Thank you so much. Fantastic work, it is amazing. We are considering to restructure the code, your pic will be a good reference.
Simple or not, the absence of the first stage makes the entire demo incomplete. The finetuing for different models maybe very different. Many beginners feel confused about it. If the authors hasnot finished stage 1 themselves, how can they say easy? If the first step has been done, is there any difficulty in sharing relevant resources?
We do not have the plan to implement stage1 soon, you can refer to https://github.com/hpcaitech/ColossalAI-Examples/tree/main/language to implement the code. We hope that more people can participate in our open source projects and contribute to us. Thanks a lot ๏ผ๏ผ
@yaleimeng Thanks for your proposal, it will take time and we have more important work now Stage 1 is a simple supervised learning process, and we have plans to add a wiki page for this project in the future so that newcomers who do not understand the field can get started quickly. We will rely more on the ecology of hugging face, so that users can focus more on the implementation of the whole RLHF process rather than the model.
