elasticsearch-perl
elasticsearch-perl copied to clipboard
late flushing
I think, this is wrong.
https://github.com/elastic/elasticsearch-perl/blob/5659396a3c6fcad01a85ecf92ff23804cf6eefe1/lib/Search/Elasticsearch/Client/7_0/Bulk.pm#L39-L50
You push something to the buffer and flush if it's greater than the limit. But shouldn't the limit be the limit, meaning: Not more than that?
Without testing I think, a correct way would be:
while (@_) {
my @json = $self->_encode_action( splice( @_, 0, 2 ) );
my $size = $self->_buffer_size;
my $added_size = 0;
$added_size += length($_) + 1 for @json;
$self->flush
if ( $max_size and $size+$added_size > $max_size ) # Would the new size exceed the max_size
|| ( $max_count and $self->_buffer_count >= $max_count ) # is the current count equal or higher than the max_count?
|| ( $max_time and time >= $self->_last_flush + $max_time );
push @$buffer, @json;
my $size = $added_size + $self->_buffer_size;
$self->_buffer_count( $self->_buffer_count + 1 );
$self->_buffer_size($size);
}
return 1;