nncf icon indicating copy to clipboard operation
nncf copied to clipboard

Use tabulate instead of texttable

Open AlexanderDokuchaev opened this issue 11 months ago • 2 comments

Changes

Using tabulate instead of texttable package to create tables.

Reason for changes

Avoid installation two packages to same task. Tabulate is optional dependencies of pandas package, that used in NNCF. https://github.com/openvinotoolkit/nncf/pull/2542

Related tickets

134603

AlexanderDokuchaev avatar Mar 13 '24 20:03 AlexanderDokuchaev

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.54%. Comparing base (1349247) to head (891fd55). Report is 1 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2574      +/-   ##
===========================================
- Coverage    90.89%   84.54%   -6.35%     
===========================================
  Files          492      492              
  Lines        45101    45096       -5     
===========================================
- Hits         40995    38128    -2867     
- Misses        4106     6968    +2862     
Files Coverage Δ
nncf/common/utils/helpers.py 100.00% <100.00%> (ø)
nncf/torch/pruning/base_algo.py 92.15% <100.00%> (-0.44%) :arrow_down:
nncf/torch/pruning/filter_pruning/algo.py 96.50% <100.00%> (ø)
setup.py 0.00% <ø> (ø)

... and 56 files with indirect coverage changes

Flag Coverage Δ
COMMON 42.77% <50.00%> (+<0.01%) :arrow_up:
ONNX 34.66% <36.36%> (+<0.01%) :arrow_up:
OPENVINO ∅ <ø> (∅)
TENSORFLOW 29.87% <45.45%> (+<0.01%) :arrow_up:
TORCH 65.76% <100.00%> (-0.01%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
common 93.11% <100.00%> (-0.65%) :arrow_down:
torch 93.59% <100.00%> (-0.01%) :arrow_down:
tensorflow 93.74% <ø> (ø)
onnx 93.05% <ø> (ø)
openvino 25.58% <ø> (-68.49%) :arrow_down:
ptq 67.31% <ø> (-20.43%) :arrow_down:

codecov[bot] avatar Mar 13 '24 20:03 codecov[bot]

Example of tables:

https://github.com/openvinotoolkit/nncf/tree/develop/examples/quantization_aware_training/torch/resnet18

+------------------+----------+-----------+-------------------------------+
|                  |   FP32   |   INT8    |            Summary            |
+==================+==========+===========+===============================+
| Accuracy@1       | 55.520   | 56.715    | 55.360 (init) + 1.355 (tuned) |
+------------------+----------+-----------+-------------------------------+
| Model Size, Mb   | 43.067   | 10.939    | Compression rate is 3.937     |
+------------------+----------+-----------+-------------------------------+
| Performance, FPS | 3743.990 | 12734.460 | Speedup x3.401                |
+------------------+----------+-----------+-------------------------------+

┍━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━┯━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┑
│                  │     FP32 │      INT8 │ Summary                       │
┝━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━┿━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┥
│ Accuracy@1       │   55.520 │    56.715 │ 55.360 (init) + 1.355 (tuned) │
├──────────────────┼──────────┼───────────┼───────────────────────────────┤
│ Model Size, Mb   │   43.067 │    10.939 │ Compression rate is 3.937     │
├──────────────────┼──────────┼───────────┼───────────────────────────────┤
│ Performance, FPS │ 3743.990 │ 12734.460 │ Speedup x3.401                │
┕━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━┷━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┙

https://github.com/openvinotoolkit/nncf/tree/develop/examples/llm_compression/openvino/tiny_llama

+--------------+---------------------------+-----------------------------------+
| Num bits (N) | % all parameters (layers) |    % ratio-defining parameters    |
|              |                           |             (layers)              |
+==============+===========================+===================================+
| 8            | 30% (20 / 156)            | 21% (18 / 154)                    |
+--------------+---------------------------+-----------------------------------+
| 4            | 70% (136 / 156)           | 79% (136 / 154)                   |
+--------------+---------------------------+-----------------------------------+

┍━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┑
│   Num bits (N) │ % all parameters (layers)   │ % ratio-defining parameters (layers)   │
┝━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┥
│              8 │ 30% (20 / 156)              │ 21% (18 / 154)                         │
├────────────────┼─────────────────────────────┼────────────────────────────────────────┤
│              4 │ 70% (136 / 156)             │ 79% (136 / 154)                        │
┕━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┙

AlexanderDokuchaev avatar Mar 14 '24 17:03 AlexanderDokuchaev