guidance

guidance copied to clipboard

Hi, I notice that the function of PEFT is still under construction. Therefore, I update guidance with PEFT. Could you please review this code and merge it? Thanks.

@HelloWorldLTY thanks, this looks great! Can you add at least one unit test to make sure this continues to work in the future? Thanks.

Sure, I will do that asap. Thanks.

Hi, I have added a unit test known as test_peft.py file. I did not test inference since it required extra files. Do I need to handle other conflict files? Thanks.

Thanks. The test currently just tests the HF code though not the new changes here :)

Also looking through this again I realized I think this also works right?

pmodel = PeftModel.from_pretrained(model, peft_model_id)
guidance.models.Transformers(pmodel, tokenizer)

If so that is probably the best way to handle this since other wise we are just rewrapping the PeftModel.from_pretrained constructor.

I think so, this is anothe alternative approach. Moreover, installing peft is still the required step I think. Do I need to modify the readme file to implement your suggestions?