pushgen [Proposal] Separating callback chain building and actual invocation

[Proposal] Separating callback chain building and actual invocation

Open NobodyXu opened this issue 3 years ago • 48 comments

Currently, the callback chain building and the actual invocation is done at the same time.

This means that invocation of Generator::run needs to check for initialization and that under debug mode, there might be 2n stack frames where n is the number of pipelines.

Here, I propose a new API for Generator:

trait Callback {
    fn call(&mut self) -> GeneratorResult;
}

trait Generator {
    type Output: Callback;

    fn run(&mut self, output: impl FnMut(Self::Output) -> crate::ValueResult) -> Self::Output;
}

Call to Generator::run will initialize the pipeline (e.g. for Dedup, it would be getting the first element) and then return a callback to a closure that have the entire pipelines already built.

As a result, even under the debug build where no inline is performed, the stack frames used is guaranteed to be equal to the number of pipelines.

The function returned will be called inside for_each, try_for_each and IteratorAdaptor.

This would also make it clear to new contributors on which part is responsible for initialization and which part is accounted for the "loop body" and possibly remove some uses of Option.

Jul 08 '21 08:07 NobodyXu

I don't think you can do a trait like that, you can't have impl FnMut() -> GeneratorResult as a function return value :(

I agree it would be nice to seperate the pipeline creation, and pipeline execution into seperate steps. However I am not sure it can be truly done. In Dedup for instance, if it fails to get the first element at build-time, it will still have to try to get it at run-time.

Say the upstream Generator is trying to fetch data from a channel, if it fails (no element to read at this time) it reports stop, if the channel is closed it reports complete, otherwise it sends each available value through the pipeline. With such a source-generator you can't rely on the first value being available when you build the pipeline. And so you must be able to do the initialization step even when you do later runs on the pipeline.

If we had a marker trait trait NeverStoppingGenerator: Generator we could move the initialization into Dedup::new for any source that is a NeverStoppingGenerator and remove that code entirely from Dedup::run when Src implements NeverStoppingGenerator, but that requires specialization I think?

Jul 08 '21 10:07 AndWass

I don't think you can do a trait like that, you can't have impl FnMut() -> GeneratorResult as a function return value :(

IMHO it should be possible, as I recalled from my memory rust's async function is just a sugar for fn f(...) -> impl Future

Say the upstream Generator is trying to fetch data from a channel, if it fails (no element to read at this time) it reports stop, if the channel is closed it reports complete, otherwise it sends each available value through the pipeline. With such a source-generator you can't rely on the first value being available when you build the pipeline. And so you must be able to do the initialization step even when you do later runs on the pipeline.

If so, then I guess the terminating for_each will just simply return I guess?

And IMO the initialization process can be restarted by calling for_each again,

It is an interesting problem though: I don't think pushgen will work well with channel, which is async by nature.

While pushgen employs callback to avoid branching, it isn't async, which I think program which uses channel would expect.

Jul 08 '21 10:07 NobodyXu

IMHO it should be possible, as I recalled from my memory rust's async function is just a sugar for fn f(...) -> impl Future

An async fn some_async() is just sugar for fn some_async() -> impl Future yes, but the Future trait itself, is alot more basic. And a quick test on the playground confirms that you can't have impl Trait as return-type in traits.

If so, then I guess the terminating for_each will just simply return I guess?

for_each will stop and return GeneratorResult::Stopped yes.

And IMO the initialization process can be restarted by calling for_each again,

True

It is an interesting problem though: I don't think pushgen will work well with channel, which is async by nature.

While pushgen employs callback to avoid branching, it isn't async, which I think program which uses channel would expect.

I was more thinking if you have a generator that uses try_recv to generate data. It might not be a common use-case but I think it can prove valuable

Jul 08 '21 10:07 AndWass

An async fn some_async() is just sugar for fn some_async() -> impl Future yes, but the Future trait itself, is alot more basic. And a quick test on the playground confirms that you can't have impl Trait as return-type in traits.

Yep, turns out only way to do this is to add an item to the Test, which implements another trait Callback since we cannot infer the type of closure.

playground

I was more thinking if you have a generator that uses try_recv to generate data. It might not be a common use-case but I think it can prove valuable

Yep, though I think my proposal will not forbid this API.

Jul 08 '21 11:07 NobodyXu

Yah I definitely think it's an interesting idea to split the cold and hot paths. Some experimentation and testing is necessary before committing to it though

Jul 08 '21 11:07 AndWass

What kind of experiments do you think is necessary for this?

Jul 08 '21 11:07 NobodyXu

I was just trying to define a clear example for the new trait, however I immediately discover that without impl FnMut() -> GeneratorResult as the return value, the code failed apart because the output is a generic parameter and it is necessary for run to return different types based on the generic parameter.

Jul 08 '21 12:07 NobodyXu

Ah ok. I will try and think of something to try as well, but right now I don't have any clear idea.

Jul 08 '21 12:07 AndWass

I just realized that all pipelines either takes &mut to itself (like Dedup) or takes nothing (Cloned).

So what I now proposed instead is to use an erased interface:

use core::ffi::c_void;
use core::mem::transmute;
use core::ptr::null_mut;
use core::marker::PhantomData;

/// ErasedFnPointer can either points to a free function or associated one that
/// `&mut self`
struct ErasedFnPointer<'a, T, Ret> {
    struct_pointer: *mut c_void,
    wrapper: fn(&ErasedFnPointer<'a, T, Ret>, T) -> Ret, 
    fp: *const (),
    phantom: PhantomData<&'a ()>
}
impl<'a, T, Ret> ErasedFnPointer<'a, T, Ret> {
    fn associated_wrapper<S>(&self, param: T) -> Ret {
        let sp = self.struct_pointer as *mut S;
        let s = unsafe { &mut (*sp) };
        let fp = unsafe { transmute::<_, fn(&mut S, T) -> Ret>(self.fp) };
        fp(s, param)
    }
    
    pub fn from_associated<S>(struct_pointer: &'a mut S, fp: fn(&mut S, T) -> Ret)
        -> ErasedFnPointer<'a, T, Ret>
    {
        ErasedFnPointer {
            struct_pointer: struct_pointer as *mut _ as *mut c_void,
            fp: fp as *const (),
            wrapper: ErasedFnPointer::associated_wrapper::<S>,
            phantom: PhantomData
        }
    }
    
    fn free_wrapper(&self, param: T) -> Ret {
        let fp = unsafe { transmute::<_, fn(T) -> Ret>(self.fp) };
        fp(param)
    }
    
    pub fn from_free(fp: fn(T) -> Ret) -> ErasedFnPointer<'static, T, Ret> {
        ErasedFnPointer {
            struct_pointer: null_mut(),
            fp: fp as *const(),
            wrapper: ErasedFnPointer::free_wrapper,
            phantom: PhantomData
        }
    }
    
    pub fn call(&self, param: T) -> Ret {
        (self.wrapper)(self, param)
    }
}

trait Callback {
    fn call(&mut self) -> GeneratorResult;
}

trait Generator {
    type Output: Callback;

    fn run(&mut self, output: ErasedFnPointer<Self::Output, crate::ValueResult>) -> Self::Output;
}

The compiler can easily inline the *_wrapper and the underlying function pointer, while Generator can have a stable interface without any generic parameter, so that it can return a single type only.

This makes the separation I described before possible without losing performance, but it also might help with compilation time a bit since there is no need for monomorphization and no duplicate code is generated when one pipeline uses the other more than once (e.x. Dedup).

playground

Edit:

~~One problem with ErasedFnPointer is that it doesn’t have PhantomData<&S> field, so it might actually caused a problem for enforcing the mutability rule.~~

I think I got it fixed now with lifetime annotation and PhantomData.

Jul 11 '21 13:07 NobodyXu

I'm not sold at the moment to be honest. This is IMO starting to get more complicated than the system already in place. Have you implemented any of the more involved adaptors (like Dedup or Flatten) using this pattern? I think before any change is decided on the basic benchmarks must be ported to this kind of system and checked against current system.

It is also quite easy to run into UB with the pointer solution; playground

Jul 11 '21 14:07 AndWass

Hi

I'm not sold at the moment to be honest. This is IMO starting to get more complicated than the system already in place. Have you implemented any of the more involved adaptors (like Dedup or Flatten) using this pattern? I think before any change is decided on the basic benchmarks must be ported to this kind of system and checked against current system.

It is also quite easy to run into UB with the pointer solution; playground

About the UB part, I have actually updated my code yesterday and introduces lifetime annotation inside ErasedFnPointer along with PhantomData to fix the lifetime issues.

With the updated code, it will actually fail to compile.

Edit:

Looks like I forgot to update the link yesterday and now I have updated the link in my original comment.

Jul 12 '21 03:07 NobodyXu

It’s actually possible to implement take one easily with closure, since in rust, associated functions are just regular functions:

fn main() {
    let mut x = None;
    ErasedFnPointer::from_associated(&mut x, |x, param| {
        *x = Some(param);
        println!("{:#?}", x);
    }).call(1);
}

Jul 12 '21 03:07 NobodyXu

For the part where deduped values is pushed, I guess the parameter output (the callback) in Generator::run has to be saved inside Dedup so that it can be accessed.

However, it does raise issues with lifetime, unless GeneratorExt::for_each and GeneratorExt::try_for_each takes self instead of &mut self.

Personally, I don’t think of this as a major downside, as users can always wraps the pipelines in a closure to use it multiple times, but it does impact existing uses of pushgen.

Jul 12 '21 03:07 NobodyXu

Here’s a link to the code inside gist

Jul 13 '21 12:07 NobodyXu

Thanks for the writeup! At the moment I won't pursue this split. I don't really see the benefit right now and I don't really have the time to do any benchmarking and testing.

Jul 13 '21 18:07 AndWass

Thanks for the writeup! At the moment I won't pursue this split. I don't really see the benefit right now and I don't really have the time to do any benchmarking and testing.

I can definitely understand your decision.

So instead of requesting for implement this in the main repo, I will try to implement this in my own fork and provides update to it in this issue.

Jul 14 '21 02:07 NobodyXu

Sounds like a good plan! I'm definitely interested in the approach.

Jul 14 '21 06:07 AndWass

I have created a new branch GeneratorImprovement for this in my own fork.

I am still in the progress of replacing the generic param inn Generator::run with ErasedFnPointer.

Jul 14 '21 13:07 NobodyXu

I have replaced the generic version of Generator::run with the ErasedFnPointer version of it in here.

I will pause for a while and perform benchmark to see how this alone would affect the performance.

Jul 15 '21 05:07 NobodyXu

Benchmark result of 6e3af98000d680f023428a5e390c573f218c8e4f using taskset 0x01 cargo bench:

    Finished bench [optimized] target(s) in 0.02s
     Running unittests (target/release/deps/pushgen-9dbcdc1345ab781d)

running 16 tests
test generator_ext::tests::for_each_stopped ... ignored
test structs::chain::tests::basic_chain ... ignored
test structs::dedup::tests::dedup_all_duplicate ... ignored
test structs::dedup::tests::dedup_nonduplicate ... ignored
test structs::dedup::tests::dedup_some_duplicate ... ignored
test structs::dedup::tests::dedup_stopping_source ... ignored
test structs::flatten::tests::slice_flatten ... ignored
test structs::flatten::tests::stopping_generator ... ignored
test structs::flatten::tests::vector_flatten ... ignored
test structs::iterator::tests::fold ... ignored
test structs::iterator::tests::iter_over_slice ... ignored
test structs::take::tests::take ... ignored
test structs::take::tests::take_restart ... ignored
test structs::zip::tests::same_length ... ignored
test structs::zip::tests::shorter_left_side ... ignored
test structs::zip::tests::shorter_right_side ... ignored

test result: ok. 0 passed; 0 failed; 16 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests (target/release/deps/for_each_chain_take_filter_map-c2f1d7bc7957186f)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_chain_take_filter_map
Benchmarking iterator_for_each_chain_take_filter_map: Warming up for 3.0000 s
Benchmarking iterator_for_each_chain_take_filter_map: Collecting 100 samples in estimated 5.1368 s (2400 iterations)
Benchmarking iterator_for_each_chain_take_filter_map: Analyzing
iterator_for_each_chain_take_filter_map
                        time:   [2.1270 ms 2.1460 ms 2.1693 ms]
                        change: [-1.2152% -0.0224% +1.1434%] (p = 0.97 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

     Running unittests (target/release/deps/for_each_dedup_filter-0792b8997f8ad6ba)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_dedup_filter
Benchmarking iterator_for_each_dedup_filter: Warming up for 3.0000 s
Benchmarking iterator_for_each_dedup_filter: Collecting 100 samples in estimated 5.3478 s (71k iterations)
Benchmarking iterator_for_each_dedup_filter: Analyzing
iterator_for_each_dedup_filter
                        time:   [74.325 us 74.800 us 75.396 us]
                        change: [-8.2384% -7.1125% -5.9268%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/for_each_dedup_flatten_filter_map-02530f85fb28b946)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_dedup_flatten_filter_map
Benchmarking iterator_for_each_dedup_flatten_filter_map: Warming up for 3.0000 s
Benchmarking iterator_for_each_dedup_flatten_filter_map: Collecting 100 samples in estimated 6.5590 s (15k iterations)
Benchmarking iterator_for_each_dedup_flatten_filter_map: Analyzing
iterator_for_each_dedup_flatten_filter_map
                        time:   [436.31 us 439.38 us 442.92 us]
                        change: [-0.4690% +0.4396% +1.3767%] (p = 0.35 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/for_each_filter_map-23889c5657b80085)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_filter_map
Benchmarking iterator_for_each_filter_map: Warming up for 3.0000 s
Benchmarking iterator_for_each_filter_map: Collecting 100 samples in estimated 5.6040 s (35k iterations)
Benchmarking iterator_for_each_filter_map: Analyzing
iterator_for_each_filter_map
                        time:   [158.13 us 159.36 us 160.81 us]
                        change: [-0.9971% +0.1623% +1.3754%] (p = 0.80 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) high mild
  7 (7.00%) high severe

     Running unittests (target/release/deps/for_each_flatten_dedup_filter_map-982b60bd65247325)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_flatten_dedup_filter_map
Benchmarking iterator_for_each_flatten_dedup_filter_map: Warming up for 3.0000 s
Benchmarking iterator_for_each_flatten_dedup_filter_map: Collecting 100 samples in estimated 6.0028 s (10k iterations)
Benchmarking iterator_for_each_flatten_dedup_filter_map: Analyzing
iterator_for_each_flatten_dedup_filter_map
                        time:   [591.43 us 595.06 us 599.53 us]
                        change: [-0.9254% +0.6120% +2.8122%] (p = 0.56 > 0.05)
                        No change in performance detected.
Found 15 outliers among 100 measurements (15.00%)
  6 (6.00%) high mild
  9 (9.00%) high severe

     Running unittests (target/release/deps/for_each_transrangers_test6-ebafe03edf02d334)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_transrangers_test6
Benchmarking iterator_for_each_transrangers_test6: Warming up for 3.0000 s
Benchmarking iterator_for_each_transrangers_test6: Collecting 100 samples in estimated 5.7450 s (20k iterations)
Benchmarking iterator_for_each_transrangers_test6: Analyzing
iterator_for_each_transrangers_test6
                        time:   [282.04 us 283.94 us 286.32 us]
                        change: [-2.7423% -1.3120% +0.0465%] (p = 0.06 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe

     Running unittests (target/release/deps/iter_next_basic_loop-37d9035efa073309)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_basic_loop
Benchmarking iterator_next_basic_loop: Warming up for 3.0000 s
Benchmarking iterator_next_basic_loop: Collecting 100 samples in estimated 5.0034 s (56k iterations)
Benchmarking iterator_next_basic_loop: Analyzing
iterator_next_basic_loop
                        time:   [89.551 us 90.250 us 91.122 us]
                        change: [-1.5815% -0.5493% +0.5786%] (p = 0.33 > 0.05)
                        No change in performance detected.
Found 21 outliers among 100 measurements (21.00%)
  12 (12.00%) low mild
  3 (3.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/iter_next_chain_take_filter_map-f319c0ad5d149fc6)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_chain_take_filter_map
Benchmarking iterator_next_chain_take_filter_map: Warming up for 3.0000 s
Benchmarking iterator_next_chain_take_filter_map: Collecting 100 samples in estimated 5.0800 s (2200 iterations)
Benchmarking iterator_next_chain_take_filter_map: Analyzing
iterator_next_chain_take_filter_map
                        time:   [2.2886 ms 2.3026 ms 2.3198 ms]
                        change: [-1.4603% -0.4064% +0.6751%] (p = 0.47 > 0.05)
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high severe

     Running unittests (target/release/deps/iter_next_dedup_filter-d972ca417e800e75)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_dedup_filter
Benchmarking iterator_next_dedup_filter: Warming up for 3.0000 s
Benchmarking iterator_next_dedup_filter: Collecting 100 samples in estimated 5.1261 s (106k iterations)
Benchmarking iterator_next_dedup_filter: Analyzing
iterator_next_dedup_filter
                        time:   [47.918 us 48.296 us 48.747 us]
                        change: [-0.2984% +0.7532% +1.8544%] (p = 0.19 > 0.05)
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high severe

     Running unittests (target/release/deps/iter_next_dedup_flatten_filter_map-b66194b56c578199)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_dedup_flatten_filter_map
Benchmarking iterator_next_dedup_flatten_filter_map: Warming up for 3.0000 s
Benchmarking iterator_next_dedup_flatten_filter_map: Collecting 100 samples in estimated 8.4989 s (10k iterations)
Benchmarking iterator_next_dedup_flatten_filter_map: Analyzing
iterator_next_dedup_flatten_filter_map
                        time:   [833.15 us 837.99 us 843.64 us]
                        change: [-0.3999% +0.8304% +2.2878%] (p = 0.23 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) low mild
  9 (9.00%) high severe

     Running unittests (target/release/deps/iter_next_filter_map-d184a6e09748526e)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_filter_map
Benchmarking iterator_next_filter_map: Warming up for 3.0000 s
Benchmarking iterator_next_filter_map: Collecting 100 samples in estimated 5.6615 s (15k iterations)
Benchmarking iterator_next_filter_map: Analyzing
iterator_next_filter_map
                        time:   [371.28 us 374.01 us 377.30 us]
                        change: [-1.1823% -0.1644% +0.7638%] (p = 0.75 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/iter_next_flatten_dedup_filter_map-3b82355d0e7308f8)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_flatten_dedup_filter_map
Benchmarking iterator_next_flatten_dedup_filter_map: Warming up for 3.0000 s

Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.1s, enable flat sampling, or reduce sample count to 70.
Benchmarking iterator_next_flatten_dedup_filter_map: Collecting 100 samples in estimated 5.0660 s (5050 iterations)
Benchmarking iterator_next_flatten_dedup_filter_map: Analyzing
iterator_next_flatten_dedup_filter_map
                        time:   [1.0013 ms 1.0090 ms 1.0183 ms]
                        change: [-4.0667% -0.9408% +1.1043%] (p = 0.61 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe

     Running unittests (target/release/deps/iter_next_transrangers_test6-aa5b656484c1d22f)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_transrangers_test6
Benchmarking iterator_next_transrangers_test6: Warming up for 3.0000 s
Benchmarking iterator_next_transrangers_test6: Collecting 100 samples in estimated 6.7045 s (10k iterations)
Benchmarking iterator_next_transrangers_test6: Analyzing
iterator_next_transrangers_test6
                        time:   [660.66 us 665.06 us 670.30 us]
                        change: [-0.6146% +0.3795% +1.4175%] (p = 0.51 > 0.05)
                        No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) high mild
  8 (8.00%) high severe

     Running unittests (target/release/deps/pushgen_chain_take_filter_map-d0156bcf4d3bfeef)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking generator_chain_take_filter_map
Benchmarking generator_chain_take_filter_map: Warming up for 3.0000 s
Benchmarking generator_chain_take_filter_map: Collecting 100 samples in estimated 7.5320 s (10k iterations)
Benchmarking generator_chain_take_filter_map: Analyzing
generator_chain_take_filter_map
                        time:   [741.44 us 746.07 us 751.54 us]
                        change: [-0.7264% +0.1440% +0.9947%] (p = 0.77 > 0.05)
                        No change in performance detected.
Found 12 outliers among 100 measurements (12.00%)
  1 (1.00%) low mild
  4 (4.00%) high mild
  7 (7.00%) high severe

     Running unittests (target/release/deps/pushgen_dedup_filter-91859256e6e92e85)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_dedup_filter
Benchmarking pushgen_dedup_filter: Warming up for 3.0000 s
Benchmarking pushgen_dedup_filter: Collecting 100 samples in estimated 5.0323 s (242k iterations)
Benchmarking pushgen_dedup_filter: Analyzing
pushgen_dedup_filter    time:   [20.762 us 20.907 us 21.089 us]
                        change: [-0.3892% +0.7025% +1.8623%] (p = 0.27 > 0.05)
                        No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/pushgen_dedup_flatten_filter_map-d78681cad649eb0f)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_dedup_flatten_filter_map
Benchmarking pushgen_dedup_flatten_filter_map: Warming up for 3.0000 s
Benchmarking pushgen_dedup_flatten_filter_map: Collecting 100 samples in estimated 5.0888 s (20k iterations)
Benchmarking pushgen_dedup_flatten_filter_map: Analyzing
pushgen_dedup_flatten_filter_map
                        time:   [250.01 us 251.90 us 254.27 us]
                        change: [-10.144% -3.6279% +0.9031%] (p = 0.30 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

     Running unittests (target/release/deps/pushgen_filter_map-9ac271e2acbf21cd)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_filter_map
Benchmarking pushgen_filter_map: Warming up for 3.0000 s
Benchmarking pushgen_filter_map: Collecting 100 samples in estimated 5.6505 s (35k iterations)
Benchmarking pushgen_filter_map: Analyzing
pushgen_filter_map      time:   [158.86 us 160.08 us 161.54 us]
                        change: [-2.1761% -0.2834% +1.2589%] (p = 0.78 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/pushgen_flatten_dedup_filter_map-64ff7169b0b1e6da)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_flatten_dedup_filter_map
Benchmarking pushgen_flatten_dedup_filter_map: Warming up for 3.0000 s
Benchmarking pushgen_flatten_dedup_filter_map: Collecting 100 samples in estimated 6.0091 s (25k iterations)
Benchmarking pushgen_flatten_dedup_filter_map: Analyzing
pushgen_flatten_dedup_filter_map
                        time:   [236.26 us 237.76 us 239.60 us]
                        change: [-0.8371% -0.0583% +0.7147%] (p = 0.89 > 0.05)
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high severe

     Running unittests (target/release/deps/pushgen_iter_basic_loop-ba604406bd167963)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_iter_basic_loop
Benchmarking pushgen_iter_basic_loop: Warming up for 3.0000 s
Benchmarking pushgen_iter_basic_loop: Collecting 100 samples in estimated 5.0472 s (56k iterations)
Benchmarking pushgen_iter_basic_loop: Analyzing
pushgen_iter_basic_loop time:   [90.251 us 90.951 us 91.823 us]
                        change: [-0.1168% +1.1477% +2.4304%] (p = 0.08 > 0.05)
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high severe

     Running unittests (target/release/deps/pushgen_iter_chain_take_filter_map-2c85753845747283)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_iter_chain_take_filter_map
Benchmarking pushgen_iter_chain_take_filter_map: Warming up for 3.0000 s

Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.8s, enable flat sampling, or reduce sample count to 50.
Benchmarking pushgen_iter_chain_take_filter_map: Collecting 100 samples in estimated 7.7878 s (5050 iterations)
Benchmarking pushgen_iter_chain_take_filter_map: Analyzing
pushgen_iter_chain_take_filter_map
                        time:   [1.4744 ms 1.4850 ms 1.4973 ms]
                        change: [-0.7726% +0.6280% +1.9490%] (p = 0.38 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe

     Running unittests (target/release/deps/pushgen_transrangers_test6-beceb9a7f7f3aa7e)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_transrangers_test6
Benchmarking pushgen_transrangers_test6: Warming up for 3.0000 s
Benchmarking pushgen_transrangers_test6: Collecting 100 samples in estimated 5.7536 s (20k iterations)
Benchmarking pushgen_transrangers_test6: Analyzing
pushgen_transrangers_test6
                        time:   [283.25 us 285.00 us 287.21 us]
                        change: [-0.9785% +0.0567% +1.1057%] (p = 0.92 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/try_for_each_chain_take_filter_map-dd6018e8ec5d4a7e)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_chain_take_filter_map
Benchmarking iterator_try_for_each_chain_take_filter_map: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_chain_take_filter_map: Collecting 100 samples in estimated 5.0208 s (2300 iterations)
Benchmarking iterator_try_for_each_chain_take_filter_map: Analyzing
iterator_try_for_each_chain_take_filter_map
                        time:   [2.1682 ms 2.1847 ms 2.2045 ms]
                        change: [-0.9323% +0.1446% +1.2684%] (p = 0.80 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

     Running unittests (target/release/deps/try_for_each_dedup_filter-e3981107a8d0b5c7)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_dedup_filter
Benchmarking iterator_try_for_each_dedup_filter: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_dedup_filter: Collecting 100 samples in estimated 5.1125 s (121k iterations)
Benchmarking iterator_try_for_each_dedup_filter: Analyzing
iterator_try_for_each_dedup_filter
                        time:   [41.759 us 42.030 us 42.379 us]
                        change: [-1.5309% -0.4606% +0.5479%] (p = 0.41 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe

     Running unittests (target/release/deps/try_for_each_dedup_flatten_filter_map-98eef37b8662f178)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_dedup_flatten_filter_map
Benchmarking iterator_try_for_each_dedup_flatten_filter_map: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_dedup_flatten_filter_map: Collecting 100 samples in estimated 6.2948 s (10k iterations)
Benchmarking iterator_try_for_each_dedup_flatten_filter_map: Analyzing
iterator_try_for_each_dedup_flatten_filter_map
                        time:   [619.25 us 623.73 us 629.01 us]
                        change: [-1.0925% -0.0937% +0.8407%] (p = 0.86 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) high mild
  7 (7.00%) high severe

     Running unittests (target/release/deps/try_for_each_filter_map-9dead610a7585c29)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_filter_map
Benchmarking iterator_try_for_each_filter_map: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_filter_map: Collecting 100 samples in estimated 6.1342 s (10k iterations)
Benchmarking iterator_try_for_each_filter_map: Analyzing
iterator_try_for_each_filter_map
                        time:   [601.05 us 604.95 us 609.77 us]
                        change: [-1.4356% -0.4445% +0.5049%] (p = 0.39 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  5 (5.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/try_for_each_flatten_dedup_filter_map-092f2984d109ffc4)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_flatten_dedup_filter_map
Benchmarking iterator_try_for_each_flatten_dedup_filter_map: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_flatten_dedup_filter_map: Collecting 100 samples in estimated 9.4996 s (10k iterations)
Benchmarking iterator_try_for_each_flatten_dedup_filter_map: Analyzing
iterator_try_for_each_flatten_dedup_filter_map
                        time:   [937.58 us 942.78 us 948.65 us]
                        change: [-5.5575% -1.8861% +0.5081%] (p = 0.33 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) high mild
  9 (9.00%) high severe

     Running unittests (target/release/deps/try_for_each_transrangers_test6-33f87431201c709c)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_transrangers_test6
Benchmarking iterator_try_for_each_transrangers_test6: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_transrangers_test6: Collecting 100 samples in estimated 7.0154 s (10k iterations)
Benchmarking iterator_try_for_each_transrangers_test6: Analyzing
iterator_try_for_each_transrangers_test6
                        time:   [690.28 us 695.47 us 701.61 us]
                        change: [-18.268% -12.381% -7.8956%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  2 (2.00%) high mild
  10 (10.00%) high severe

Jul 15 '21 06:07 NobodyXu

Benchmark of 6090ec39757f7cc85eb6c1583482c40f41d6223f:

   Compiling pushgen v0.0.1 (/home/nobodyxu/Dev/pushgen)
    Finished bench [optimized] target(s) in 3m 03s
     Running unittests (target/release/deps/pushgen-9dbcdc1345ab781d)

running 19 tests
test callback::tests::from_associated ... ignored
test callback::tests::from_associated_for_closure ... ignored
test callback::tests::from_free ... ignored
test generator_ext::tests::for_each_stopped ... ignored
test structs::chain::tests::basic_chain ... ignored
test structs::dedup::tests::dedup_all_duplicate ... ignored
test structs::dedup::tests::dedup_nonduplicate ... ignored
test structs::dedup::tests::dedup_some_duplicate ... ignored
test structs::dedup::tests::dedup_stopping_source ... ignored
test structs::flatten::tests::slice_flatten ... ignored
test structs::flatten::tests::stopping_generator ... ignored
test structs::flatten::tests::vector_flatten ... ignored
test structs::iterator::tests::fold ... ignored
test structs::iterator::tests::iter_over_slice ... ignored
test structs::take::tests::take ... ignored
test structs::take::tests::take_restart ... ignored
test structs::zip::tests::same_length ... ignored
test structs::zip::tests::shorter_left_side ... ignored
test structs::zip::tests::shorter_right_side ... ignored

test result: ok. 0 passed; 0 failed; 19 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests (target/release/deps/for_each_chain_take_filter_map-c2f1d7bc7957186f)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_chain_take_filter_map
Benchmarking iterator_for_each_chain_take_filter_map: Warming up for 3.0000 s
Benchmarking iterator_for_each_chain_take_filter_map: Collecting 100 samples in estimated 5.1416 s (2400 iterations)
Benchmarking iterator_for_each_chain_take_filter_map: Analyzing
iterator_for_each_chain_take_filter_map
                        time:   [2.1307 ms 2.1469 ms 2.1664 ms]
                        change: [-1.2470% +0.0425% +1.3581%] (p = 0.94 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

     Running unittests (target/release/deps/for_each_dedup_filter-0792b8997f8ad6ba)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_dedup_filter
Benchmarking iterator_for_each_dedup_filter: Warming up for 3.0000 s
Benchmarking iterator_for_each_dedup_filter: Collecting 100 samples in estimated 5.0360 s (66k iterations)
Benchmarking iterator_for_each_dedup_filter: Analyzing
iterator_for_each_dedup_filter
                        time:   [76.366 us 77.038 us 77.825 us]
                        change: [+1.7262% +3.4645% +5.2383%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe

     Running unittests (target/release/deps/for_each_dedup_flatten_filter_map-02530f85fb28b946)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_dedup_flatten_filter_map
Benchmarking iterator_for_each_dedup_flatten_filter_map: Warming up for 3.0000 s
Benchmarking iterator_for_each_dedup_flatten_filter_map: Collecting 100 samples in estimated 6.5673 s (15k iterations)
Benchmarking iterator_for_each_dedup_flatten_filter_map: Analyzing
iterator_for_each_dedup_flatten_filter_map
                        time:   [427.84 us 430.80 us 434.27 us]
                        change: [-2.5552% -1.6524% -0.6604%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/for_each_filter_map-23889c5657b80085)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_filter_map
Benchmarking iterator_for_each_filter_map: Warming up for 3.0000 s
Benchmarking iterator_for_each_filter_map: Collecting 100 samples in estimated 5.6767 s (35k iterations)
Benchmarking iterator_for_each_filter_map: Analyzing
iterator_for_each_filter_map
                        time:   [159.62 us 160.85 us 162.31 us]
                        change: [-0.2898% +0.9400% +2.0048%] (p = 0.14 > 0.05)
                        No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
  2 (2.00%) high mild
  7 (7.00%) high severe

     Running unittests (target/release/deps/for_each_flatten_dedup_filter_map-982b60bd65247325)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_flatten_dedup_filter_map
Benchmarking iterator_for_each_flatten_dedup_filter_map: Warming up for 3.0000 s
Benchmarking iterator_for_each_flatten_dedup_filter_map: Collecting 100 samples in estimated 6.0153 s (10k iterations)
Benchmarking iterator_for_each_flatten_dedup_filter_map: Analyzing
iterator_for_each_flatten_dedup_filter_map
                        time:   [591.23 us 595.11 us 599.90 us]
                        change: [-2.7574% -0.8490% +0.5473%] (p = 0.39 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/for_each_transrangers_test6-ebafe03edf02d334)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_transrangers_test6
Benchmarking iterator_for_each_transrangers_test6: Warming up for 3.0000 s
Benchmarking iterator_for_each_transrangers_test6: Collecting 100 samples in estimated 5.7753 s (20k iterations)
Benchmarking iterator_for_each_transrangers_test6: Analyzing
iterator_for_each_transrangers_test6
                        time:   [283.04 us 284.92 us 287.28 us]
                        change: [-0.5860% +0.3632% +1.3297%] (p = 0.47 > 0.05)
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high severe

     Running unittests (target/release/deps/iter_next_basic_loop-37d9035efa073309)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_basic_loop
Benchmarking iterator_next_basic_loop: Warming up for 3.0000 s
Benchmarking iterator_next_basic_loop: Collecting 100 samples in estimated 5.3813 s (61k iterations)
Benchmarking iterator_next_basic_loop: Analyzing
iterator_next_basic_loop
                        time:   [88.227 us 88.830 us 89.586 us]
                        change: [-2.3394% -1.1818% -0.0575%] (p = 0.05 < 0.05)
                        Change within noise threshold.
Found 12 outliers among 100 measurements (12.00%)
  7 (7.00%) high mild
  5 (5.00%) high severe

     Running unittests (target/release/deps/iter_next_chain_take_filter_map-f319c0ad5d149fc6)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_chain_take_filter_map
Benchmarking iterator_next_chain_take_filter_map: Warming up for 3.0000 s
Benchmarking iterator_next_chain_take_filter_map: Collecting 100 samples in estimated 5.0784 s (2200 iterations)
Benchmarking iterator_next_chain_take_filter_map: Analyzing
iterator_next_chain_take_filter_map
                        time:   [2.2954 ms 2.3124 ms 2.3330 ms]
                        change: [-0.5810% +0.4244% +1.5850%] (p = 0.45 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/iter_next_dedup_filter-d972ca417e800e75)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_dedup_filter
Benchmarking iterator_next_dedup_filter: Warming up for 3.0000 s
Benchmarking iterator_next_dedup_filter: Collecting 100 samples in estimated 5.0968 s (106k iterations)
Benchmarking iterator_next_dedup_filter: Analyzing
iterator_next_dedup_filter
                        time:   [47.792 us 48.176 us 48.636 us]
                        change: [-1.4840% -0.3737% +0.6225%] (p = 0.51 > 0.05)
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) high mild
  4 (4.00%) high severe

     Running unittests (target/release/deps/iter_next_dedup_flatten_filter_map-b66194b56c578199)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_dedup_flatten_filter_map
Benchmarking iterator_next_dedup_flatten_filter_map: Warming up for 3.0000 s
Benchmarking iterator_next_dedup_flatten_filter_map: Collecting 100 samples in estimated 8.5047 s (10k iterations)
Benchmarking iterator_next_dedup_flatten_filter_map: Analyzing
iterator_next_dedup_flatten_filter_map
                        time:   [840.23 us 845.14 us 850.85 us]
                        change: [-1.0482% +0.2435% +1.2954%] (p = 0.71 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) high mild
  9 (9.00%) high severe

     Running unittests (target/release/deps/iter_next_filter_map-d184a6e09748526e)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_filter_map
Benchmarking iterator_next_filter_map: Warming up for 3.0000 s
Benchmarking iterator_next_filter_map: Collecting 100 samples in estimated 5.6721 s (15k iterations)
Benchmarking iterator_next_filter_map: Analyzing
iterator_next_filter_map
                        time:   [371.88 us 374.77 us 378.24 us]
                        change: [-0.5513% +0.3788% +1.5762%] (p = 0.49 > 0.05)
                        No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/iter_next_flatten_dedup_filter_map-3b82355d0e7308f8)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_flatten_dedup_filter_map
Benchmarking iterator_next_flatten_dedup_filter_map: Warming up for 3.0000 s

Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.1s, enable flat sampling, or reduce sample count to 60.
Benchmarking iterator_next_flatten_dedup_filter_map: Collecting 100 samples in estimated 5.0808 s (5050 iterations)
Benchmarking iterator_next_flatten_dedup_filter_map: Analyzing
iterator_next_flatten_dedup_filter_map
                        time:   [994.51 us 1.0027 ms 1.0125 ms]
                        change: [-2.0237% -0.9274% +0.0606%] (p = 0.09 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe

     Running unittests (target/release/deps/iter_next_transrangers_test6-aa5b656484c1d22f)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_transrangers_test6
Benchmarking iterator_next_transrangers_test6: Warming up for 3.0000 s
Benchmarking iterator_next_transrangers_test6: Collecting 100 samples in estimated 6.7035 s (10k iterations)
Benchmarking iterator_next_transrangers_test6: Analyzing
iterator_next_transrangers_test6
                        time:   [660.18 us 664.37 us 669.43 us]
                        change: [-1.4956% -0.3559% +0.7084%] (p = 0.55 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/pushgen_chain_take_filter_map-d0156bcf4d3bfeef)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking generator_chain_take_filter_map
Benchmarking generator_chain_take_filter_map: Warming up for 3.0000 s
Benchmarking generator_chain_take_filter_map: Collecting 100 samples in estimated 6.4890 s (10k iterations)
Benchmarking generator_chain_take_filter_map: Analyzing
generator_chain_take_filter_map
                        time:   [638.75 us 643.78 us 649.60 us]
                        change: [-14.694% -13.955% -13.188%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) high mild
  7 (7.00%) high severe

     Running unittests (target/release/deps/pushgen_dedup_filter-91859256e6e92e85)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_dedup_filter
Benchmarking pushgen_dedup_filter: Warming up for 3.0000 s
Benchmarking pushgen_dedup_filter: Collecting 100 samples in estimated 5.0207 s (242k iterations)
Benchmarking pushgen_dedup_filter: Analyzing
pushgen_dedup_filter    time:   [20.577 us 20.747 us 20.954 us]
                        change: [-2.3968% -1.0662% +0.1615%] (p = 0.10 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/pushgen_dedup_flatten_filter_map-d78681cad649eb0f)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_dedup_flatten_filter_map
Benchmarking pushgen_dedup_flatten_filter_map: Warming up for 3.0000 s
Benchmarking pushgen_dedup_flatten_filter_map: Collecting 100 samples in estimated 5.1514 s (10k iterations)
Benchmarking pushgen_dedup_flatten_filter_map: Analyzing
pushgen_dedup_flatten_filter_map
                        time:   [506.45 us 510.29 us 514.96 us]
                        change: [+99.196% +101.67% +103.99%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) low mild
  5 (5.00%) high severe

     Running unittests (target/release/deps/pushgen_filter_map-9ac271e2acbf21cd)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_filter_map
Benchmarking pushgen_filter_map: Warming up for 3.0000 s
Benchmarking pushgen_filter_map: Collecting 100 samples in estimated 5.6642 s (35k iterations)
Benchmarking pushgen_filter_map: Analyzing
pushgen_filter_map      time:   [159.16 us 160.20 us 161.49 us]
                        change: [-1.1014% +0.1143% +1.2986%] (p = 0.86 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/pushgen_flatten_dedup_filter_map-64ff7169b0b1e6da)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_flatten_dedup_filter_map
Benchmarking pushgen_flatten_dedup_filter_map: Warming up for 3.0000 s
Benchmarking pushgen_flatten_dedup_filter_map: Collecting 100 samples in estimated 8.9666 s (10k iterations)
Benchmarking pushgen_flatten_dedup_filter_map: Analyzing
pushgen_flatten_dedup_filter_map
                        time:   [885.01 us 890.09 us 895.92 us]
                        change: [+271.62% +274.84% +278.06%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) high mild
  9 (9.00%) high severe

     Running unittests (target/release/deps/pushgen_iter_basic_loop-ba604406bd167963)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_iter_basic_loop
Benchmarking pushgen_iter_basic_loop: Warming up for 3.0000 s

Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.5s, enable flat sampling, or reduce sample count to 50.
Benchmarking pushgen_iter_basic_loop: Collecting 100 samples in estimated 7.5461 s (5050 iterations)
Benchmarking pushgen_iter_basic_loop: Analyzing
pushgen_iter_basic_loop time:   [1.4848 ms 1.4939 ms 1.5045 ms]
                        change: [+1525.7% +1544.2% +1561.7%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
  3 (3.00%) high mild
  9 (9.00%) high severe

     Running unittests (target/release/deps/pushgen_iter_chain_take_filter_map-2c85753845747283)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_iter_chain_take_filter_map
Benchmarking pushgen_iter_chain_take_filter_map: Warming up for 3.0000 s

Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.7s, enable flat sampling, or reduce sample count to 60.
Benchmarking pushgen_iter_chain_take_filter_map: Collecting 100 samples in estimated 6.7268 s (5050 iterations)
Benchmarking pushgen_iter_chain_take_filter_map: Analyzing
pushgen_iter_chain_take_filter_map
                        time:   [1.3233 ms 1.3336 ms 1.3458 ms]
                        change: [-11.833% -10.562% -9.1405%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

     Running unittests (target/release/deps/pushgen_transrangers_test6-beceb9a7f7f3aa7e)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_transrangers_test6
Benchmarking pushgen_transrangers_test6: Warming up for 3.0000 s
Benchmarking pushgen_transrangers_test6: Collecting 100 samples in estimated 7.5459 s (10k iterations)
Benchmarking pushgen_transrangers_test6: Analyzing
pushgen_transrangers_test6
                        time:   [743.61 us 748.33 us 753.88 us]
                        change: [+159.79% +162.52% +165.39%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) high mild
  9 (9.00%) high severe

     Running unittests (target/release/deps/try_for_each_chain_take_filter_map-dd6018e8ec5d4a7e)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_chain_take_filter_map
Benchmarking iterator_try_for_each_chain_take_filter_map: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_chain_take_filter_map: Collecting 100 samples in estimated 5.0196 s (2300 iterations)
Benchmarking iterator_try_for_each_chain_take_filter_map: Analyzing
iterator_try_for_each_chain_take_filter_map
                        time:   [2.1683 ms 2.1860 ms 2.2080 ms]
                        change: [-1.1223% +0.0580% +1.3757%] (p = 0.93 > 0.05)
                        No change in performance detected.
Found 16 outliers among 100 measurements (16.00%)
  6 (6.00%) low mild
  4 (4.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/try_for_each_dedup_filter-e3981107a8d0b5c7)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_dedup_filter
Benchmarking iterator_try_for_each_dedup_filter: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_dedup_filter: Collecting 100 samples in estimated 5.0968 s (121k iterations)
Benchmarking iterator_try_for_each_dedup_filter: Analyzing
iterator_try_for_each_dedup_filter
                        time:   [41.793 us 42.111 us 42.503 us]
                        change: [-0.8095% +0.1412% +1.1413%] (p = 0.78 > 0.05)
                        No change in performance detected.
Found 12 outliers among 100 measurements (12.00%)
  6 (6.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/try_for_each_dedup_flatten_filter_map-98eef37b8662f178)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_dedup_flatten_filter_map
Benchmarking iterator_try_for_each_dedup_flatten_filter_map: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_dedup_flatten_filter_map: Collecting 100 samples in estimated 6.2973 s (10k iterations)
Benchmarking iterator_try_for_each_dedup_flatten_filter_map: Analyzing
iterator_try_for_each_dedup_flatten_filter_map
                        time:   [619.80 us 623.83 us 628.69 us]
                        change: [-0.8105% +0.0665% +0.9953%] (p = 0.88 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

     Running unittests (target/release/deps/try_for_each_filter_map-9dead610a7585c29)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_filter_map
Benchmarking iterator_try_for_each_filter_map: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_filter_map: Collecting 100 samples in estimated 6.1253 s (10k iterations)
Benchmarking iterator_try_for_each_filter_map: Analyzing
iterator_try_for_each_filter_map
                        time:   [602.46 us 606.62 us 611.60 us]
                        change: [-0.7174% +0.1216% +1.0620%] (p = 0.79 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  2 (2.00%) high mild
  8 (8.00%) high severe

     Running unittests (target/release/deps/try_for_each_flatten_dedup_filter_map-092f2984d109ffc4)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_flatten_dedup_filter_map
Benchmarking iterator_try_for_each_flatten_dedup_filter_map: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_flatten_dedup_filter_map: Collecting 100 samples in estimated 9.5406 s (10k iterations)
Benchmarking iterator_try_for_each_flatten_dedup_filter_map: Analyzing
iterator_try_for_each_flatten_dedup_filter_map
                        time:   [937.79 us 943.25 us 949.28 us]
                        change: [-1.2269% -0.2153% +0.6648%] (p = 0.67 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) high mild
  10 (10.00%) high severe

     Running unittests (target/release/deps/try_for_each_transrangers_test6-33f87431201c709c)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_transrangers_test6
Benchmarking iterator_try_for_each_transrangers_test6: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_transrangers_test6: Collecting 100 samples in estimated 7.0399 s (10k iterations)
Benchmarking iterator_try_for_each_transrangers_test6: Analyzing
iterator_try_for_each_transrangers_test6
                        time:   [692.75 us 697.97 us 704.16 us]
                        change: [-0.4898% +0.8592% +2.2790%] (p = 0.25 > 0.05)
                        No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
  1 (1.00%) high mild
  12 (12.00%) high severe

Jul 15 '21 06:07 NobodyXu

Interesting!

Looks like there were performance-regression for pretty much all benchmarks except for chain_take_filter_map? I wonder why that managed to improve while the others regressed by such hefty amounts?

Jul 15 '21 06:07 AndWass

It seems benchmark test

iterator_for_each_dedup_flatten_filter_map
iterator_next_basic_loop
generator_chain_take_filter_map
pushgen_iter_chain_take_filter_map

are showing improvements, not just chain_take_filter_map.

Jul 15 '21 06:07 NobodyXu

@AndWass How about also running the benchmark on your computer to confirm these benchmark tests are indeed showing improvements?

Jul 15 '21 06:07 NobodyXu

It seems benchmark test iterator_for_each_dedup_flatten_filter_map, iterator_next_basic_loop, generator_chain_take_filter_map and pushgen_iter_chain_take_filter_map are showing improvements, not just chain_take_filter_map.

The first two are not using pushgen at all so shouldn't be impacted by the changes. The improvement percentage is also small, so more likely that there was some jitter during those benchmarks.

The other two benchmark the same operations using pushgen and the iterator adaptor of pushgen. So they are closely related.

Jul 15 '21 06:07 AndWass

@AndWass How about also running the benchmark on your computer to confirm these benchmark tests are indeed showing improvements?

I'll try and run them tonight.

Jul 15 '21 06:07 AndWass

It seems benchmark test iterator_for_each_dedup_flatten_filter_map, iterator_next_basic_loop, generator_chain_take_filter_map and pushgen_iter_chain_take_filter_map are showing improvements, not just chain_take_filter_map.

The first two are not using pushgen at all so shouldn't be impacted by the changes. The improvement percentage is also small, so more likely that there was some jitter during those benchmarks.

The other two benchmark the same operations using pushgen and the iterator adaptor of pushgen. So they are closely related.

Ah, sorry that I got them wrong.

Jul 15 '21 07:07 NobodyXu

Ah, sorry that I got them wrong.

Nothing to be sorry about!

Jul 15 '21 07:07 AndWass

BTW, I also did another benchmark for b949ebaace966a5f3417ccca3edc0e80a5a2bf32 which adds #[inline(always)] to associated functions of ErasedFnPointer:

NOTE that the I ran this benchmark after the previous one on GeneratorImprovement branch, so the result here isn't relative to main.

   Compiling pushgen v0.0.1 (/home/nobodyxu/Dev/pushgen)
    Finished bench [optimized] target(s) in 3m 04s
     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/pushgen-9dbcdc1345ab781d)

running 19 tests
test callback::tests::from_associated ... ignored
test callback::tests::from_associated_for_closure ... ignored
test callback::tests::from_free ... ignored
test generator_ext::tests::for_each_stopped ... ignored
test structs::chain::tests::basic_chain ... ignored
test structs::dedup::tests::dedup_all_duplicate ... ignored
test structs::dedup::tests::dedup_nonduplicate ... ignored
test structs::dedup::tests::dedup_some_duplicate ... ignored
test structs::dedup::tests::dedup_stopping_source ... ignored
test structs::flatten::tests::slice_flatten ... ignored
test structs::flatten::tests::stopping_generator ... ignored
test structs::flatten::tests::vector_flatten ... ignored
test structs::iterator::tests::fold ... ignored
test structs::iterator::tests::iter_over_slice ... ignored
test structs::take::tests::take ... ignored
test structs::take::tests::take_restart ... ignored
test structs::zip::tests::same_length ... ignored
test structs::zip::tests::shorter_left_side ... ignored
test structs::zip::tests::shorter_right_side ... ignored

test result: ok. 0 passed; 0 failed; 19 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/for_each_chain_take_filter_map-c2f1d7bc7957186f)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_chain_take_filter_map
Benchmarking iterator_for_each_chain_take_filter_map: Warming up for 3.0000 s
Benchmarking iterator_for_each_chain_take_filter_map: Collecting 100 samples in estimated 5.1524 s (2400 iterations)
Benchmarking iterator_for_each_chain_take_filter_map: Analyzing
iterator_for_each_chain_take_filter_map
                        time:   [2.1299 ms 2.1443 ms 2.1619 ms]
                        change: [-1.1988% -0.1180% +1.0000%] (p = 0.84 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) high mild
  6 (6.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/for_each_dedup_filter-0792b8997f8ad6ba)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_dedup_filter
Benchmarking iterator_for_each_dedup_filter: Warming up for 3.0000 s
Benchmarking iterator_for_each_dedup_filter: Collecting 100 samples in estimated 5.1628 s (66k iterations)
Benchmarking iterator_for_each_dedup_filter: Analyzing
iterator_for_each_dedup_filter
                        time:   [74.275 us 74.873 us 75.595 us]
                        change: [-5.6235% -3.8748% -2.4623%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) high mild
  6 (6.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/for_each_dedup_flatten_filter_map-02530f85fb28b946)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_dedup_flatten_filter_map
Benchmarking iterator_for_each_dedup_flatten_filter_map: Warming up for 3.0000 s
Benchmarking iterator_for_each_dedup_flatten_filter_map: Collecting 100 samples in estimated 6.5704 s (15k iterations)
Benchmarking iterator_for_each_dedup_flatten_filter_map: Analyzing
iterator_for_each_dedup_flatten_filter_map
                        time:   [432.76 us 435.92 us 439.57 us]
                        change: [+0.0471% +1.0949% +2.1850%] (p = 0.04 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  7 (7.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/for_each_filter_map-23889c5657b80085)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_filter_map
Benchmarking iterator_for_each_filter_map: Warming up for 3.0000 s
Benchmarking iterator_for_each_filter_map: Collecting 100 samples in estimated 5.6405 s (35k iterations)
Benchmarking iterator_for_each_filter_map: Analyzing
iterator_for_each_filter_map
                        time:   [158.40 us 159.49 us 160.81 us]
                        change: [-2.0810% -1.0343% -0.0990%] (p = 0.04 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  6 (6.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/for_each_flatten_dedup_filter_map-982b60bd65247325)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_flatten_dedup_filter_map
Benchmarking iterator_for_each_flatten_dedup_filter_map: Warming up for 3.0000 s
Benchmarking iterator_for_each_flatten_dedup_filter_map: Collecting 100 samples in estimated 6.0156 s (10k iterations)
Benchmarking iterator_for_each_flatten_dedup_filter_map: Analyzing
iterator_for_each_flatten_dedup_filter_map
                        time:   [591.22 us 595.44 us 600.52 us]
                        change: [-0.8741% +0.0444% +0.9715%] (p = 0.93 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) high mild
  6 (6.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/for_each_transrangers_test6-ebafe03edf02d334)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_for_each_transrangers_test6
Benchmarking iterator_for_each_transrangers_test6: Warming up for 3.0000 s
Benchmarking iterator_for_each_transrangers_test6: Collecting 100 samples in estimated 5.7434 s (20k iterations)
Benchmarking iterator_for_each_transrangers_test6: Analyzing
iterator_for_each_transrangers_test6
                        time:   [283.30 us 285.40 us 287.92 us]
                        change: [-0.8300% +0.3013% +1.5750%] (p = 0.65 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  7 (7.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/iter_next_basic_loop-37d9035efa073309)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_basic_loop
Benchmarking iterator_next_basic_loop: Warming up for 3.0000 s
Benchmarking iterator_next_basic_loop: Collecting 100 samples in estimated 5.0658 s (56k iterations)
Benchmarking iterator_next_basic_loop: Analyzing
iterator_next_basic_loop
                        time:   [90.813 us 91.433 us 92.212 us]
                        change: [+0.9357% +2.0819% +3.2069%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/iter_next_chain_take_filter_map-f319c0ad5d149fc6)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_chain_take_filter_map
Benchmarking iterator_next_chain_take_filter_map: Warming up for 3.0000 s
Benchmarking iterator_next_chain_take_filter_map: Collecting 100 samples in estimated 5.0712 s (2200 iterations)
Benchmarking iterator_next_chain_take_filter_map: Analyzing
iterator_next_chain_take_filter_map
                        time:   [2.2794 ms 2.2967 ms 2.3175 ms]
                        change: [-1.8310% -0.6786% +0.4489%] (p = 0.26 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/iter_next_dedup_filter-d972ca417e800e75)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_dedup_filter
Benchmarking iterator_next_dedup_filter: Warming up for 3.0000 s
Benchmarking iterator_next_dedup_filter: Collecting 100 samples in estimated 5.0780 s (106k iterations)
Benchmarking iterator_next_dedup_filter: Analyzing
iterator_next_dedup_filter
                        time:   [47.163 us 47.527 us 47.966 us]
                        change: [-1.6487% +0.2363% +2.6709%] (p = 0.87 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  6 (6.00%) high mild
  5 (5.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/iter_next_dedup_flatten_filter_map-b66194b56c578199)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_dedup_flatten_filter_map
Benchmarking iterator_next_dedup_flatten_filter_map: Warming up for 3.0000 s
Benchmarking iterator_next_dedup_flatten_filter_map: Collecting 100 samples in estimated 8.2807 s (10k iterations)
Benchmarking iterator_next_dedup_flatten_filter_map: Analyzing
iterator_next_dedup_flatten_filter_map
                        time:   [824.25 us 829.92 us 836.17 us]
                        change: [-3.1964% -2.3549% -1.5474%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  6 (6.00%) high mild
  2 (2.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/iter_next_filter_map-d184a6e09748526e)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_filter_map
Benchmarking iterator_next_filter_map: Warming up for 3.0000 s
Benchmarking iterator_next_filter_map: Collecting 100 samples in estimated 5.6743 s (15k iterations)
Benchmarking iterator_next_filter_map: Analyzing
iterator_next_filter_map
                        time:   [371.77 us 374.19 us 377.11 us]
                        change: [-1.3948% -0.2976% +0.7256%] (p = 0.59 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  5 (5.00%) high mild
  6 (6.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/iter_next_flatten_dedup_filter_map-3b82355d0e7308f8)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_flatten_dedup_filter_map
Benchmarking iterator_next_flatten_dedup_filter_map: Warming up for 3.0000 s

Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.1s, enable flat sampling, or reduce sample count to 70.
Benchmarking iterator_next_flatten_dedup_filter_map: Collecting 100 samples in estimated 5.0779 s (5050 iterations)
Benchmarking iterator_next_flatten_dedup_filter_map: Analyzing
iterator_next_flatten_dedup_filter_map
                        time:   [999.23 us 1.0070 ms 1.0166 ms]
                        change: [-0.5148% +0.4892% +1.4748%] (p = 0.35 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  6 (6.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/iter_next_transrangers_test6-aa5b656484c1d22f)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_next_transrangers_test6
Benchmarking iterator_next_transrangers_test6: Warming up for 3.0000 s
Benchmarking iterator_next_transrangers_test6: Collecting 100 samples in estimated 6.6861 s (10k iterations)
Benchmarking iterator_next_transrangers_test6: Analyzing
iterator_next_transrangers_test6
                        time:   [659.96 us 664.28 us 669.30 us]
                        change: [-1.0315% +0.0658% +1.3197%] (p = 0.92 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) high mild
  7 (7.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/pushgen_chain_take_filter_map-d0156bcf4d3bfeef)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking generator_chain_take_filter_map
Benchmarking generator_chain_take_filter_map: Warming up for 3.0000 s
Benchmarking generator_chain_take_filter_map: Collecting 100 samples in estimated 7.5446 s (10k iterations)
Benchmarking generator_chain_take_filter_map: Analyzing
generator_chain_take_filter_map
                        time:   [742.68 us 747.65 us 753.43 us]
                        change: [+15.287% +16.276% +17.305%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) high mild
  7 (7.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/pushgen_dedup_filter-91859256e6e92e85)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_dedup_filter
Benchmarking pushgen_dedup_filter: Warming up for 3.0000 s
Benchmarking pushgen_dedup_filter: Collecting 100 samples in estimated 5.0660 s (247k iterations)
Benchmarking pushgen_dedup_filter: Analyzing
pushgen_dedup_filter    time:   [20.463 us 20.620 us 20.816 us]
                        change: [-1.7907% -0.6087% +0.6338%] (p = 0.34 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/pushgen_dedup_flatten_filter_map-d78681cad649eb0f)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_dedup_flatten_filter_map
Benchmarking pushgen_dedup_flatten_filter_map: Warming up for 3.0000 s
Benchmarking pushgen_dedup_flatten_filter_map: Collecting 100 samples in estimated 5.1006 s (20k iterations)
Benchmarking pushgen_dedup_flatten_filter_map: Analyzing
pushgen_dedup_flatten_filter_map
                        time:   [250.08 us 251.93 us 254.27 us]
                        change: [-50.976% -50.378% -49.703%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  6 (6.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/pushgen_filter_map-9ac271e2acbf21cd)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_filter_map
Benchmarking pushgen_filter_map: Warming up for 3.0000 s
Benchmarking pushgen_filter_map: Collecting 100 samples in estimated 5.6106 s (35k iterations)
Benchmarking pushgen_filter_map: Analyzing
pushgen_filter_map      time:   [157.58 us 158.76 us 160.16 us]
                        change: [-2.2154% -1.1452% -0.0369%] (p = 0.05 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  8 (8.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/pushgen_flatten_dedup_filter_map-64ff7169b0b1e6da)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_flatten_dedup_filter_map
Benchmarking pushgen_flatten_dedup_filter_map: Warming up for 3.0000 s
Benchmarking pushgen_flatten_dedup_filter_map: Collecting 100 samples in estimated 6.0200 s (25k iterations)
Benchmarking pushgen_flatten_dedup_filter_map: Analyzing
pushgen_flatten_dedup_filter_map
                        time:   [237.16 us 238.80 us 240.80 us]
                        change: [-73.441% -73.118% -72.731%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) high mild
  7 (7.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/pushgen_iter_basic_loop-ba604406bd167963)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_iter_basic_loop
Benchmarking pushgen_iter_basic_loop: Warming up for 3.0000 s
Benchmarking pushgen_iter_basic_loop: Collecting 100 samples in estimated 5.4069 s (61k iterations)
Benchmarking pushgen_iter_basic_loop: Analyzing
pushgen_iter_basic_loop time:   [88.381 us 89.033 us 89.833 us]
                        change: [-94.090% -94.038% -93.987%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) high mild
  6 (6.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/pushgen_iter_chain_take_filter_map-2c85753845747283)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_iter_chain_take_filter_map
Benchmarking pushgen_iter_chain_take_filter_map: Warming up for 3.0000 s

Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.8s, enable flat sampling, or reduce sample count to 50.
Benchmarking pushgen_iter_chain_take_filter_map: Collecting 100 samples in estimated 7.7925 s (5050 iterations)
Benchmarking pushgen_iter_chain_take_filter_map: Analyzing
pushgen_iter_chain_take_filter_map
                        time:   [1.5369 ms 1.5456 ms 1.5559 ms]
                        change: [+13.369% +14.945% +16.368%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low mild
  7 (7.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/pushgen_transrangers_test6-beceb9a7f7f3aa7e)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking pushgen_transrangers_test6
Benchmarking pushgen_transrangers_test6: Warming up for 3.0000 s
Benchmarking pushgen_transrangers_test6: Collecting 100 samples in estimated 5.7539 s (20k iterations)
Benchmarking pushgen_transrangers_test6: Analyzing
pushgen_transrangers_test6
                        time:   [283.17 us 285.38 us 288.03 us]
                        change: [-62.310% -61.886% -61.430%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/try_for_each_chain_take_filter_map-dd6018e8ec5d4a7e)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_chain_take_filter_map
Benchmarking iterator_try_for_each_chain_take_filter_map: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_chain_take_filter_map: Collecting 100 samples in estimated 5.0112 s (2300 iterations)
Benchmarking iterator_try_for_each_chain_take_filter_map: Analyzing
iterator_try_for_each_chain_take_filter_map
                        time:   [2.1670 ms 2.1834 ms 2.2033 ms]
                        change: [-1.3590% -0.1154% +1.1142%] (p = 0.86 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  6 (6.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/try_for_each_dedup_filter-e3981107a8d0b5c7)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_dedup_filter
Benchmarking iterator_try_for_each_dedup_filter: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_dedup_filter: Collecting 100 samples in estimated 5.1109 s (121k iterations)
Benchmarking iterator_try_for_each_dedup_filter: Analyzing
iterator_try_for_each_dedup_filter
                        time:   [41.814 us 42.134 us 42.522 us]
                        change: [-0.9225% +0.0574% +1.0324%] (p = 0.91 > 0.05)
                        No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
  2 (2.00%) high mild
  7 (7.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/try_for_each_dedup_flatten_filter_map-98eef37b8662f178)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_dedup_flatten_filter_map
Benchmarking iterator_try_for_each_dedup_flatten_filter_map: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_dedup_flatten_filter_map: Collecting 100 samples in estimated 6.2983 s (10k iterations)
Benchmarking iterator_try_for_each_dedup_flatten_filter_map: Analyzing
iterator_try_for_each_dedup_flatten_filter_map
                        time:   [619.76 us 624.02 us 629.05 us]
                        change: [-0.7038% +0.1630% +0.9587%] (p = 0.71 > 0.05)
                        No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) high mild
  8 (8.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/try_for_each_filter_map-9dead610a7585c29)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_filter_map
Benchmarking iterator_try_for_each_filter_map: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_filter_map: Collecting 100 samples in estimated 6.1309 s (10k iterations)
Benchmarking iterator_try_for_each_filter_map: Analyzing
iterator_try_for_each_filter_map
                        time:   [601.42 us 605.28 us 610.06 us]
                        change: [-0.9470% -0.0525% +0.8285%] (p = 0.91 > 0.05)
                        No change in performance detected.
Found 12 outliers among 100 measurements (12.00%)
  6 (6.00%) high mild
  6 (6.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/try_for_each_flatten_dedup_filter_map-092f2984d109ffc4)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_flatten_dedup_filter_map
Benchmarking iterator_try_for_each_flatten_dedup_filter_map: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_flatten_dedup_filter_map: Collecting 100 samples in estimated 9.4908 s (10k iterations)
Benchmarking iterator_try_for_each_flatten_dedup_filter_map: Analyzing
iterator_try_for_each_flatten_dedup_filter_map
                        time:   [935.91 us 940.68 us 946.20 us]
                        change: [-0.5845% +0.2973% +1.4541%] (p = 0.57 > 0.05)
                        No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
  2 (2.00%) low mild
  2 (2.00%) high mild
  9 (9.00%) high severe

     Running unittests (/home/nobodyxu/Dev/pushgen/target/release/deps/try_for_each_transrangers_test6-33f87431201c709c)
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.

Gnuplot not found, using plotters backend
Benchmarking iterator_try_for_each_transrangers_test6
Benchmarking iterator_try_for_each_transrangers_test6: Warming up for 3.0000 s
Benchmarking iterator_try_for_each_transrangers_test6: Collecting 100 samples in estimated 7.1084 s (10k iterations)
Benchmarking iterator_try_for_each_transrangers_test6: Analyzing
iterator_try_for_each_transrangers_test6
                        time:   [699.31 us 704.44 us 710.48 us]
                        change: [-0.7978% +0.7563% +2.2267%] (p = 0.34 > 0.05)
                        No change in performance detected.
Found 12 outliers among 100 measurements (12.00%)
  2 (2.00%) high mild
  10 (10.00%) high severe

Jul 15 '21 07:07 NobodyXu

Seems like adding #[inline(always)] does dramatically affect

pushgen_dedup_flatten_filter_map,
pushgen_filter_map,
pushgen_flatten_dedup_filter_map,
pushgen_iter_basic_loop,
pushgen_transrangers_test6

Jul 15 '21 07:07 NobodyXu

pushgen pushgen copied to clipboard

[Proposal] Separating callback chain building and actual invocation

pushgen
pushgen copied to clipboard