elasticsearch-perl
elasticsearch-perl copied to clipboard
Try::Tiny has some performance issues
So I'm using this amongst other things to load my indexes up in bulk. This process takes some 5-6 hours so I've been profiling to see where I can improve it (It's already been improved quite a bit, it was between 8 and 18 hours).
One of the things I noticed is that you're using Try::Tiny everywhere (but especially within your json encoder role).
According to https://metacpan.org/pod/Try::Tiny::Tiny and http://blogs.perl.org/users/diab_jerius/2017/04/how-fast-can-you-try.html Try::Tiny is actually pretty slow.
In Search::Elasticsearch::Role::Serializer::JSON I changed the encode routine to
#===================================
sub encode {
#===================================
my ( $self, $var ) = @_;
unless ( ref $var ) {
return is_utf8($var)
? encode_utf8($var)
: $var;
}
my $return;
eval {
$return = $self->JSON->encode($var);
};
if ($@) {
throw( "Serializer",$@, { var => $var } );
}
return $return;
}
On a limited run which took 4:22 before I changed, it now takes 3:43. This is borne out by my profiling which shows it taking 30 seconds https://imgur.com/a/7A8jUg5
Results from my changed version are shown here https://imgur.com/a/cljcPQR
I was really quite surprised that Try::Tiny had that much overhead but it may be worth considering switching back to using raw eval or Syntax::Keyword::Try which came out favourably as well.
If you like I can send a pull request in with the relevant changes. Minimized to the json encode/decode stuff if wanted. I've gone another way with c++ version for the extra speed so it's not urgent for me right now.
Sorry I haven't gotten back to you on this. Interesting find, and worth changing. I had a quick grep through the module and I'm using none of the more interesting features of Try::Tiny, so it could be replaced by pretty much any of the alternative modules. One thing I'm not sure about is if those modules play nicely with AnyEvent.
I'd be happy to review a PR, even just targeting the JSON encoding bit (but would be good to completely switch modules instead)
Syntax::Keyword::Try is also less quirky than Try::Tiny
Syntax::Keyword::Try is also less quirky than Try::Tiny
I did consider that but it has a hard requirement of perl 5.14 and it's still somewhat slower than eval, so I figured if we were speeding things up it made sense to just go with the basics. It's also much less likely to conflict with AnyEvent
Thanks for this PR and sorry for the slow responses. I'm trying to find time to release a 7.0 compatible version at the moment, and will look at merging this at the same time.
eval() is probably the way to go in terms of speed and compatibility.
@spansh, how much slower was Syntax::Keyword::Try than eval?
The link in the original post has speed comparisons http://blogs.perl.org/users/diab_jerius/2017/04/how-fast-can-you-try.html
@spansh thanks for the link, I'll have a look to your PR soon.