Liger-Kernel icon indicating copy to clipboard operation
Liger-Kernel copied to clipboard

Fix errorbar in benchmarks_visualizer.py

Open Stonepia opened this issue 7 months ago • 1 comments

Summary

The default errorbar is None, which would lead to "TypeError: unsupported operand type(s) for -: 'int' and 'NoneType'".

We don't override the errorbar and let seaborn decide the width.

When running the benchmark_visualizer.py from the tutorial, I got the following error message:

python benchmarks_visualizer.py --kernel-name rope --metric-name memory --kernel-operation-mode full  
Traceback (most recent call last):
  File "/home/usr/folder/ptenv/Liger-Kernel/benchmark/benchmarks_visualizer.py", line 164, in <module>
    main()
  File "/home/usr/folder/ptenv/Liger-Kernel/benchmark/benchmarks_visualizer.py", line 160, in main
    plot_data(df, config)
  File "/home/usr/folder/ptenv/Liger-Kernel/benchmark/benchmarks_visualizer.py", line 112, in plot_data
    ax = sns.lineplot(
         ^^^^^^^^^^^^^
  File "/home/usr/miniforge3/envs/ptenv/lib/python3.12/site-packages/seaborn/relational.py", line 515, in lineplot
    p.plot(ax, kwargs)
  File "/home/usr/miniforge3/envs/ptenv/lib/python3.12/site-packages/seaborn/relational.py", line 295, in plot
    grouped
  File "/home/usr/miniforge3/envs/ptenv/lib/python3.12/site-packages/pandas/core/groupby/groupby.py", line 1819, in apply
    return self._python_apply_general(f, self._obj_with_exclusions)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/usr/miniforge3/envs/ptenv/lib/python3.12/site-packages/pandas/core/groupby/groupby.py", line 1885, in _python_apply_general
    values, mutated = self._grouper.apply_groupwise(f, data, self.axis)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/usr/miniforge3/envs/ptenv/lib/python3.12/site-packages/pandas/core/groupby/ops.py", line 919, in apply_groupwise
    res = f(group)
          ^^^^^^^^
  File "/home/usr/miniforge3/envs/ptenv/lib/python3.12/site-packages/pandas/core/groupby/groupby.py", line 1809, in f
    return func(g, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/usr/miniforge3/envs/ptenv/lib/python3.12/site-packages/seaborn/_statistics.py", line 518, in __call__
    err_min, err_max = _percentile_interval(boots, self.error_level)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/usr/miniforge3/envs/ptenv/lib/python3.12/site-packages/seaborn/_statistics.py", line 665, in _percentile_interval
    edge = (100 - width) / 2
            ~~~~^~~~~~~
TypeError: unsupported operand type(s) for -: 'int' and 'NoneType'

Environment

python version: 3.12.10 pandas: 2.2.3 seaborn: 0.13.2 Liger Kernel: 34bbeb64c47be8a810b48a4fc8dbe9612316cdbd (May 12)

Testing Done

  • Hardware Type: <BLANK>
  • [ ] run make test to ensure correctness
  • [x] run make checkstyle to ensure code style
  • [ ] run make test-convergence to ensure convergence

Stonepia avatar May 13 '25 06:05 Stonepia

Thank you! However, for rope memory benchmark, there shouldn't exist more than one y_value on any x_values, so the plot it draws is actually incorrect.

It is because there are different extra_benchmark_config in our all_benchmark_data.csv, but the filtering logic here is insufficient: https://github.com/linkedin/Liger-Kernel/blob/a26c869fc3a519e392a527be7bd3f568451b9a32/benchmark/benchmarks_visualizer.py#L81-L90

I'll open an issue for it.

Tcc0403 avatar May 16 '25 04:05 Tcc0403