[DOC]: How to do the supervised fine-tuning (SFT) (Stage 1 of chatGPT) ？
Open
yaleimeng
opened this issue 1 year ago
•
9 comments
📚 The doc issue
We only see stage 2 and 3 in the chatpgt/example/readme file.
Will you please show us how to finetune the gpt/blooom/opt model with an example dataset ?
Simple or not, the absence of the first stage makes the entire demo incomplete. The finetuing for different models maybe very different. Many beginners feel confused about it.
If the authors hasnot finished stage 1 themselves, how can they say easy? If the first step has been done, is there any difficulty in sharing relevant resources?
I’ve spent some time drawing a diagram to illustrate the calling process, share with you all
Here is the code to fine-tune the Bloom model, petals
from petals import DistributedBloomForCausalLM
model = DistributedBloomForCausalLM.from_pretrained("bigscience/bloom-petals", tuning_mode="ptune", pre_seq_len=16)
# Embeddings & prompts are on your device, BLOOM blocks are distributed across the Internet
inputs = tokenizer("A cat sat", return_tensors="pt")["input_ids"]
outputs = model.generate(inputs, max_new_tokens=5)
print(tokenizer.decode(outputs[0])) # A cat sat on a mat...
# Fine-tuning (updates only prompts or adapters hosted locally)
optimizer = torch.optim.AdamW(model.parameters())
for input_ids, labels in data_loader:
outputs = model.forward(input_ids)
loss = cross_entropy(outputs.logits, labels)
optimizer.zero_grad()
loss.backward()
optimizer.step()
I’ve spent some time drawing a diagram to illustrate the calling process, share with you all
Here is the code to fine-tune the Bloom model, petals
from petals import DistributedBloomForCausalLM
model = DistributedBloomForCausalLM.from_pretrained("bigscience/bloom-petals", tuning_mode="ptune", pre_seq_len=16)
# Embeddings & prompts are on your device, BLOOM blocks are distributed across the Internet
inputs = tokenizer("A cat sat", return_tensors="pt")["input_ids"]
outputs = model.generate(inputs, max_new_tokens=5)
print(tokenizer.decode(outputs[0])) # A cat sat on a mat...
# Fine-tuning (updates only prompts or adapters hosted locally)
optimizer = torch.optim.AdamW(model.parameters())
for input_ids, labels in data_loader:
outputs = model.forward(input_ids)
loss = cross_entropy(outputs.logits, labels)
optimizer.zero_grad()
loss.backward()
optimizer.step()
Awesome! Thank you so much. Fantastic work, it is amazing.
We are considering to restructure the code, your pic will be a good reference.
Simple or not, the absence of the first stage makes the entire demo incomplete. The finetuing for different models maybe very different. Many beginners feel confused about it. If the authors hasnot finished stage 1 themselves, how can they say easy? If the first step has been done, is there any difficulty in sharing relevant resources?
We do not have the plan to implement stage1 soon, you can refer to https://github.com/hpcaitech/ColossalAI-Examples/tree/main/language to implement the code.
We hope that more people can participate in our open source projects and contribute to us. Thanks a lot ！！
@yaleimeng Thanks for your proposal, it will take time and we have more important work now
Stage 1 is a simple supervised learning process, and we have plans to add a wiki page for this project in the future so that newcomers who do not understand the field can get started quickly. We will rely more on the ecology of hugging face, so that users can focus more on the implementation of the whole RLHF process rather than the model.
ColossalAI copied to clipboard
hpcaitech
