elasticsearch-perl icon indicating copy to clipboard operation
elasticsearch-perl copied to clipboard

Try::Tiny has some performance issues

Open spansh opened this issue 5 years ago • 9 comments

So I'm using this amongst other things to load my indexes up in bulk. This process takes some 5-6 hours so I've been profiling to see where I can improve it (It's already been improved quite a bit, it was between 8 and 18 hours).

One of the things I noticed is that you're using Try::Tiny everywhere (but especially within your json encoder role).

According to https://metacpan.org/pod/Try::Tiny::Tiny and http://blogs.perl.org/users/diab_jerius/2017/04/how-fast-can-you-try.html Try::Tiny is actually pretty slow.

In Search::Elasticsearch::Role::Serializer::JSON I changed the encode routine to

#===================================
sub encode {
#===================================
    my ( $self, $var ) = @_;
    unless ( ref $var ) {
        return is_utf8($var)
            ? encode_utf8($var)
            : $var;
    }
    my $return;
    eval {
        $return = $self->JSON->encode($var);
    };
    if ($@) {
        throw( "Serializer",$@, { var => $var } );
    }

    return $return;
}

On a limited run which took 4:22 before I changed, it now takes 3:43. This is borne out by my profiling which shows it taking 30 seconds https://imgur.com/a/7A8jUg5

Results from my changed version are shown here https://imgur.com/a/cljcPQR

I was really quite surprised that Try::Tiny had that much overhead but it may be worth considering switching back to using raw eval or Syntax::Keyword::Try which came out favourably as well.

spansh avatar Feb 22 '19 09:02 spansh

If you like I can send a pull request in with the relevant changes. Minimized to the json encode/decode stuff if wanted. I've gone another way with c++ version for the extra speed so it's not urgent for me right now.

spansh avatar Mar 20 '19 16:03 spansh

Sorry I haven't gotten back to you on this. Interesting find, and worth changing. I had a quick grep through the module and I'm using none of the more interesting features of Try::Tiny, so it could be replaced by pretty much any of the alternative modules. One thing I'm not sure about is if those modules play nicely with AnyEvent.

I'd be happy to review a PR, even just targeting the JSON encoding bit (but would be good to completely switch modules instead)

clintongormley avatar Mar 20 '19 17:03 clintongormley

Syntax::Keyword::Try is also less quirky than Try::Tiny

djzort avatar May 01 '19 04:05 djzort

Syntax::Keyword::Try is also less quirky than Try::Tiny

I did consider that but it has a hard requirement of perl 5.14 and it's still somewhat slower than eval, so I figured if we were speeding things up it made sense to just go with the basics. It's also much less likely to conflict with AnyEvent

spansh avatar May 01 '19 08:05 spansh

Thanks for this PR and sorry for the slow responses. I'm trying to find time to release a 7.0 compatible version at the moment, and will look at merging this at the same time.

clintongormley avatar May 02 '19 07:05 clintongormley

eval() is probably the way to go in terms of speed and compatibility.

djzort avatar Jan 10 '20 03:01 djzort

@spansh, how much slower was Syntax::Keyword::Try than eval?

matthewlenz avatar Feb 28 '20 00:02 matthewlenz

The link in the original post has speed comparisons http://blogs.perl.org/users/diab_jerius/2017/04/how-fast-can-you-try.html

spansh avatar Feb 28 '20 09:02 spansh

@spansh thanks for the link, I'll have a look to your PR soon.

ezimuel avatar Mar 11 '20 22:03 ezimuel