rank_llm
rank_llm copied to clipboard
Adds script for AWQ-quantizing model
Pull Request Checklist
Reference Issue
ref: https://github.com/castorini/ura-projects/issues/4
Checklist Items
Before submitting your pull request, please review these items:
- [Yes] Have you followed the contributing guidelines?
- [Yes] Have you verified that there are no existing Pull Requests for the same update/change?
- [Yes] Have you updated any relevant documentation or added new tests where needed?
PR Type
What kind of change does this PR introduce?
- [ ] Bugfix
- [ ] Feature
- [ ] Code style update (formatting, local variables)
- [ ] Refactoring (no functional changes, no api changes)
- [ ] Documentation content changes
- [ ] Other...
- Description: Adds model quantization using AWQ