json_parse_exception via Scroll API
I'm attempting to use the scroll API against Elasticsearch v6.3.0 with Search::Elasticsearch v6.0
for my $name (@species) {
$iterable = $es->scroll_helper(
index => 'trackhubs_v1',
body => {
query => { match => {'species.scientific_name' => $name } } ,
}
);
while (my $hit = $iterable->next()) {
# extract unique stuff
}
}
This works for some iterations, and is spooling the first ten of 948 results as I would hope, but then it chokes on $iterable->next() with the following error:
With vars: {'body' => {'status' => 400,'error' => {'root_cause' => [{'reason' => 'Failed to parse request body','type' => 'illegal_argument_exception'}],'reason' => 'Failed to parse request body','type' => 'illegal_argument_exception','caused_by' => {'reason' => 'Unrecognized token \'DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAByFmZ4ZVpGXzlwUXlpYUh2b1YtcXJXQmcAAAAAAAAAdBZmeGVaRl85cFF5aWFIdm9WLXFyV0JnAAAAAAAAAHMWZnhlWkZfOXBReWlhSHZvVi1xcldCZw\': was expecting (\'true\', \'false\' or \'null\')
at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@30a9fd0; line: 1, column: 148]','type' => 'json_parse_exception'}}},
'request' => {
'serialize' => 'std',
'path' => '/_search/scroll',
'ignore' => [],
'mime_type' => 'application/json',
'body' => 'DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAByFmZ4ZVpGXzlwUXlpYUh2b1YtcXJXQmcAAAAAAAAAdBZmeGVaRl85cFF5aWFIdm9WLXFyV0JnAAAAAAAAAHMWZnhlWkZfOXBReWlhSHZvVi1xcldCZw==',
'qs' => {'scroll' => '1m'},
'method' => 'GET'
},'status_code' => 400}
Is this functionality somehow different for ES6? I note a comment about support for ES5 in the POD, but wasn't sure how seriously to take it.
The client sends these requests: GET $HOST/trackhubs_v1/_search?scroll=1m HTTP/1.1 (application/json) Response 200 OK GET $HOST/_search/scroll?scroll=1m HTTP/1.1 (application/json) Response 400 Bad
Notably, the second request has a body that is not JSON, but base64-encoded: queryThenFetchfxeZF_9pQyiaHvoV-qrWBgfxeZF_9pQyiaHvoV-qrWBgfxeZF_9pQyiaHvoV-qrWBg
I suppose something is going wrong when the scroll tries to fill its buffer? Ask if you need any more debug.