LoRA
LoRA copied to clipboard
microsoft
Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
Bumps [mistune](https://github.com/lepture/mistune) from 0.8.4 to 2.0.3. Release notes Sourced from mistune's releases. Version 2.0.2 Fix escape_url via lepture/mistune#295 Version 2.0.1 Fix XSS for image link syntax. Version 2.0.0 First release...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Please avoid using `.T` `RuntimeError: Exporting the operator numpy_T to ONNX opset version 12 is not supported. Please open a bug to request ONNX export support for the missing operator.`...
There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the...
![microsoft-github-policy-service[bot] avatar](https://avatars.githubusercontent.com/in/95686?v=4 "microsoft-github-policy-service[bot] avatar"){kind=avatar}
Please accept this contribution adding the standard Microsoft SECURITY.MD :lock: file to help the community understand the security policy and how to safely report security issues. GitHub uses the presence...
I'm trying to switch all my embedding and Linear layers by Lora layers. Although the GPU size needed reduces, the training time remains the same, even with less trainable weights....
Bumps [numpy](https://github.com/numpy/numpy) from 1.19.2 to 1.22.0. Release notes Sourced from numpy's releases. v1.22.0 NumPy 1.22.0 Release Notes NumPy 1.22.0 is a big release featuring the work of 153 contributors spread...
Bumps [notebook](http://jupyter.org) from 6.4.1 to 6.4.12. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) You can trigger a rebase of this PR by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands...
Hi, For LoRA/loralib/layers.py line 71, should we use nn.Embedding.eval() instead of nn.Linear.eval() here. 
Bumps [pillow](https://github.com/python-pillow/Pillow) from 8.3.2 to 9.0.1. Release notes Sourced from pillow's releases. 9.0.1 https://pillow.readthedocs.io/en/stable/releasenotes/9.0.1.html Changes In show_file, use os.remove to remove temporary images. CVE-2022-24303 #6010 [@radarhere, @hugovk] Restrict builtins within...
Hi, I really enjoy the work you propose! In learning the paper and the code, I have a question regarding the implementation of GPT2LMModel's forward function (https://github.com/microsoft/LoRA/blob/aa68d8a021c7ba08973e35fdfdc76338fdbfad57/examples/NLG/src/model.py#L396). I notice the...
