neural-compressor
neural-compressor copied to clipboard
Published
20 hours ago •
intel
intel
Support save/load API for WOQ
Type of Change
feature
Description
Support save/load API for WOQ remove export_compressed_model from config
Expected Behavior & Potential Risk
the expected behavior that triggered by this PR
How has this PR been tested?
UT local test: fp32&rtn
| ks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| lambada_openai | 1 | none | 0 | perplexity | 26.0209 | ± | 0.9382 |
| none | 0 | acc | 0.3790 | ± | 0.0068 |
Accuracy: 0.37900 Batch size = 1
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| lambada_openai | 1 | none | 0 | perplexity | 29.1191 | ± | 1.1134 |
| none | 0 | acc | 0.3679 | ± | 0.0067 |
Accuracy: 0.36794 Batch size = 1
opt_125m_woq_gptq_int4_dq_bnb
| ks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| lambada_openai | 1 | none | 0 | perplexity | 26.9172 | ± | 1.0165 |
| none | 0 | acc | 0.3701 | ± | 0.0067 | ||
| Accuracy: 0.37008 | |||||||
| Batch size = 1 |
Dependency Change?
any library dependency introduced or removed
