LoRA

LoRA copied to clipboard

Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.0.4 to 6.3.3. Changelog Sourced from tornado's changelog. Release notes .. toctree:: :maxdepth: 2 releases/v6.3.3 releases/v6.3.2 releases/v6.3.1 releases/v6.3.0 releases/v6.2.0 releases/v6.1.0 releases/v6.0.4 releases/v6.0.3 releases/v6.0.2 releases/v6.0.1 releases/v6.0.0 releases/v5.1.1 releases/v5.1.0...

dependabot[bot]

RuntimeError: CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling `cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)`

4

Hi! I am trying to use LoRA for my convolution layers: `self.conv = Conv2d(1, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)`. I used lora counterpart of` nn. Conv2D` as`lora.Conv2d(n_chans_in,...

aleemsidra

How to apply LoRA for TTS models

1

Hi, I've been trying to apply LoRA to the VITS model (hence the pull request for the conv1d). Turns out just using Lora for the text encoder transformer isn't enough,...

nivibilla

get state dict OOM

1

I train llama 13 in 8 3090 with lora. Model can be forwarded and backwarded. But when model get state dict, gpu is OOM.

WallE-Chang

M1 Pro 14 inch

1

Does lora can be run on M1 Pro MacBook? 14gb gpu

farhan-niloy

Replicating Result on WebNLG

2

Thanks for your nice work. I am try to replicate result on webNLG, but the finnal epochs of checkpoint is only 11270, different from 20000. This results in a significant...

vvhj

the pretrained-layer's param has been re-init

5

hi, I use the 'from_pretrain ' func to load the pretrain model ,but I found the linear param will be re-init when I simply replace the nn.Linear with lora.Linear

CuddleSabe

How does MergedLinear work?

7

I understand why we need MergedLinear but is there a simple example of how the forward pass works for a MergedLinear? Specifically this line -> https://github.com/microsoft/LoRA/blob/main/loralib/layers.py#L248. I'm struggling to understand...

tspeterkim

[info] Tensorflow based open-source implementations

1

Are there any TF implementations available that you are aware of? Also, do you see any specific limitations in converting this repo to TF?

arunbaby0

Some errors in your generation code, but if you use the code from huggingface, it works well

7

Dear Edward, Thanks for your contribution to the community. But I couldn't re-implement your experiments by using the scripts you wrote in the LoRA/examples/NLG. I feel down and don't know...

zibojia