elasticsearch-perl icon indicating copy to clipboard operation
elasticsearch-perl copied to clipboard

json_parse_exception via Scroll API

Open nerdstrike opened this issue 7 years ago • 0 comments

I'm attempting to use the scroll API against Elasticsearch v6.3.0 with Search::Elasticsearch v6.0

for my $name (@species) {
  $iterable = $es->scroll_helper(
      index => 'trackhubs_v1',
      body => {
        query => { match => {'species.scientific_name' => $name } } ,
      }
    );

  while (my $hit = $iterable->next()) {
      # extract unique stuff
  }
}

This works for some iterations, and is spooling the first ten of 948 results as I would hope, but then it chokes on $iterable->next() with the following error:

With vars: {'body' => {'status' => 400,'error' => {'root_cause' => [{'reason' => 'Failed to parse request body','type' => 'illegal_argument_exception'}],'reason' => 'Failed to parse request body','type' => 'illegal_argument_exception','caused_by' => {'reason' => 'Unrecognized token \'DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAByFmZ4ZVpGXzlwUXlpYUh2b1YtcXJXQmcAAAAAAAAAdBZmeGVaRl85cFF5aWFIdm9WLXFyV0JnAAAAAAAAAHMWZnhlWkZfOXBReWlhSHZvVi1xcldCZw\': was expecting (\'true\', \'false\' or \'null\')
 at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@30a9fd0; line: 1, column: 148]','type' => 'json_parse_exception'}}},
'request' => {
  'serialize' => 'std',
  'path' => '/_search/scroll',
  'ignore' => [],
  'mime_type' => 'application/json',
  'body' => 'DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAByFmZ4ZVpGXzlwUXlpYUh2b1YtcXJXQmcAAAAAAAAAdBZmeGVaRl85cFF5aWFIdm9WLXFyV0JnAAAAAAAAAHMWZnhlWkZfOXBReWlhSHZvVi1xcldCZw==',
  'qs' => {'scroll' => '1m'},
  'method' => 'GET'
},'status_code' => 400}

Is this functionality somehow different for ES6? I note a comment about support for ES5 in the POD, but wasn't sure how seriously to take it.

The client sends these requests: GET $HOST/trackhubs_v1/_search?scroll=1m HTTP/1.1 (application/json) Response 200 OK GET $HOST/_search/scroll?scroll=1m HTTP/1.1 (application/json) Response 400 Bad

Notably, the second request has a body that is not JSON, but base64-encoded: queryThenFetchŠfxeZF_9pQyiaHvoV-qrWBg‹fxeZF_9pQyiaHvoV-qrWBgŒfxeZF_9pQyiaHvoV-qrWBg

I suppose something is going wrong when the scroll tries to fill its buffer? Ask if you need any more debug.

nerdstrike avatar Jul 17 '18 13:07 nerdstrike