Criterion is unable to generate line chart when BenchmarkId parameter is not number

[Jianqoq]

[Jianqoq]

for the line chart, when axis is a string or other type implemented display, line chart won't be generated

[grimerssy]

Seems logical to me. How should the parameters be placed on the line chart axis then? If you want to compare benchmarks with different inputs, I'd say that the violin plot is sufficient

[Jianqoq]

for example, if I am doing a computation of N dimension array, the input is shape like [0, 1, 2], [0, 1], etc. I will want to see how the performance gose compare with other people's implementation. the Y axis value will be time

[grimerssy]

To do that you still can use the violin plot that is generated even if the BenchmarkId parameter is a string. And line chart is impossible to make with arbitrary sized arrays since there is no way to properly organise them on the axis. If it was possible there would be a way for you to tell if array A is as far away on the axis from array B as B is from C

[Jianqoq]

The shape I pass to is a string, violin is not straight forward, people can't immediately understand it. line chart is more straight forward.

[frostyplanet]

I agree, It's useful to threat string (display) as x axis

[frostyplanet]

I am unable to generate line chart in report. even I change benchmarkId to u64, it still does not show up.

fn bench_tokio(c: &mut Criterion) {
    let mut group = c.benchmark_group("tokio_mpsc");
    group.significance_level(0.1).sample_size(50);
    group.measurement_time(Duration::from_secs(10));
    for input in [1, 2, 4, 8, 16] {
        group.throughput(Throughput::Elements(TOTAL_MESSAGES as u64));
        group.bench_with_input(BenchmarkId::new("mpsc", input as u64), &input, |b, i| {
            b.to_async(get_runtime()).iter(|| _tokio_bounded_mpsc(100, input))
        });
    }
}