zenml-projects icon indicating copy to clipboard operation
zenml-projects copied to clipboard

Use QDoRA instead of just LoRA

Open strickvl opened this issue 6 months ago • 1 comments

Uses QDoRA instead of just LoRA. This has some efficiency benefits, notably faster training without a corresponding performance loss. See this for more details.

The code on the base branch was also unformatted, but (aside from formatting changes) I've highlighted the specific lines where I've actually changed code.

strickvl avatar Aug 22 '24 11:08 strickvl