statistex icon indicating copy to clipboard operation
statistex copied to clipboard

Outlier detection and outlier exclusion

Open NickNeck opened this issue 1 year ago • 1 comments

Hello @PragTob

This PR adds outlier handling to statistex. The goal of this changes is to use the new functionality in benchee to solve issue https://github.com/bencheeorg/benchee/issues/382

A little test with benchee, this PR and the benchmark from the readme in benchee show the following effect:

outlier exclusion: none
Name                  ips        average  deviation         median         99th %
flat_map           4.35 K      229.88 μs     ±9.86%      226.75 μs      406.74 μs
map.flatten        2.43 K      410.68 μs    ±15.94%      410.71 μs      567.22 μs

outlier exclusion: once
Name                  ips        average  deviation         median         99th %
flat_map           4.41 K      226.95 μs     ±0.15%      226.92 μs      228.17 μs
map.flatten        2.42 K      413.41 μs     ±1.50%      412.21 μs      436.42 μs

outlier exclusion: repeatedly
Name                  ips        average  deviation         median         99th %
flat_map           4.40 K      227.21 μs     ±0.10%      227.21 μs      227.79 μs
map.flatten        2.43 K      411.59 μs     ±0.82%      411.42 μs      419.54 μs

I have also made a change in Statistex.Percentiles. The function percentiles expects now a sorted list and the Statistex module is responsible for doing the sort. This change is to prevent the list from being sorted more than once.

WDYT? and thanks for all the work on benchee.

NickNeck avatar Jul 04 '23 21:07 NickNeck

:wave: Thank you a ton, I have seen this and have some time these days and I'll get to this in the coming days I hope :) And sorry for the looong wait :(

Thanks for tackling this major issue and feature! :green_heart:

IMG_20220602_080206

PragTob avatar Nov 10 '23 11:11 PragTob