openvino
openvino copied to clipboard
[GPU] Add Validate pass call after IncreasePositionIdsPrecision to ensure proper data type propagation
Details:
This patch adds Validate pass call after IncreasePositionIdsPrecision to ensure proper data type propagation
With this change the accuracy of llama-3-8b INT8 (and other LLMs probably) can be restored to expected level Before:
| Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
|--------|------:|------|-----:|---------------|---|-----:|---|------|
|wikitext| 2|none | 0|bits_per_byte |↓ |0.6030|± |N/A |
| | |none | 0|byte_perplexity|↓ |1.5189|± |N/A |
| | |none | 0|word_perplexity|↓ |9.3472|± |N/A |
After:
| Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
|--------|------:|------|-----:|---------------|---|-----:|---|------|
|wikitext| 2|none | 0|bits_per_byte |↓ |0.5351|± |N/A |
| | |none | 0|byte_perplexity|↓ |1.4490|± |N/A |
| | |none | 0|word_perplexity|↓ |7.2664|± |N/A |