criterion.rs icon indicating copy to clipboard operation
criterion.rs copied to clipboard

[BUG] Running Unusual UTF-8 Characters

Open midapsh opened this issue 2 years ago • 1 comments

Issue

Hi all,

I'm having issues running a benchmark for a slugify like library (e.g., python-slugify and slugify). I've created a fork from the project slug-rs and tried to run a bench against it. But, when it finishs, it shows that the test has not happened.

Thanks in advance.

Computer Hardware Specifications

  • OS Name: Microsoft Windows 11 Pro
  • Version: 10.0.22000 Build 22000
  • System Type: x64-based PC
  • Processor: 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz, 2304 Mhz, 8 Core(s), 16 Logical Processor(s)
  • Locale: Brazil
  • Time Zone: E. South America Standard Time
  • Installed Physical Memory (RAM): 16.0 GB
  • Total Physical Memory: 15.7 GB
  • Available Physical Memory: 3.49 GB
  • Total Virtual Memory: 30.0 GB
  • Available Virtual Memory: 4.94 GB

Registry Editor Specifications

Computer\HKEY_CURRENT_USER\Control Panel\Desktop\LanguageConfiguration

  • Name: (Default)
  • Type: REG_SZ
  • Data: (value not set)

Computer\HKEY_CURRENT_USER\Control Panel\Desktop\LanguageConfiguration

  • Name: en-US
  • Type: REG_MULTI_SZ
  • Data: pt-BR

Computer\HKEY_CURRENT_USER\Control Panel\Desktop\LanguageConfigurationPending

  • Name: (Default)
  • Type: REG_SZ
  • Data: (value not set)

Computer\HKEY_CURRENT_USER\Control Panel\Desktop\LanguageConfigurationPending

  • Name: pt-BR
  • Type: REG_MULTI_SZ
  • Data: en-US

midapsh avatar Jun 26 '22 15:06 midapsh

Hello!

I cloned your repository and ran your benchmarks and they seem to run about as well as I'd expect. They're defined incorrectly though:

c.bench_function("bench_slug", |b| {
        b.iter(|| black_box(slugify(black_box("My test Slug!!"))));
        b.iter(|| black_box(slugify(black_box("Test Slug2!!"))));
        b.iter(|| black_box(slugify(black_box("Æúűűűű--cool?"))));
        b.iter(|| black_box(slugify(black_box("long long long long      long"))));
    });

You shouldn't iterate a bencher more than once per benchmark. If you want to test your code on multiple inputs, those should be multiple benchmarks (perhaps using a BenchmarkGroup to collect them together). If you do this, you'll just end up overwriting the first three sets of measurements with the last one.

I also see that you're using the real_blackbox feature. If that's what you want to do, fair enough, but it's generally not necessary.

I am going to leave this open though because we should have it raise an error when misused like this.

bheisler avatar Jul 07 '22 00:07 bheisler