max_bulk_volume is not having any effect

[cabhi]

Hi @jprante

I need to do a bulk import (~ 1 billion records /10g data) from oracle to elastic search. i used following settings

"max_bulk_actions" : "200000", "max_concurrent_bulk_requests" : "50", "max_bulk_volume": "100mb",

i can see these values printed in logs also

[20:17:36,853][INFO ][org.xbib.elasticsearch.helper.client.BaseTransportClient][pool-2-thread-1] creating transport client on Linux OpenJDK 64-Bit Server VM Oracle Corporation 1.8.0_91-8u91-b14-0ubuntu4~14.04-b14 25.91-b14 with effective settings {autodiscover=false, client.transport.ignore_cluster_name=false, client.transport.nodes_sampler_interval=5s, client.transport.ping_timeout=5s, cluster.name=data-load, flush_interval=5s, host.0=localhost, max_actions_per_request=200000, max_concurrent_requests=50, max_volume_per_request=100mb, name=importer, port=9300, sniff=false}

but when i look at transport logs it showing me ~10mb bytes only --

[20:36:50,800][DEBUG][org.xbib.elasticsearch.helper.client.BulkTransportClient][elasticsearch[importer][bulk_processor][T#1]] before bulk [202] [actions=37887] [bytes=12477885] [concurrent requests=1] [20:36:56,341][DEBUG][org.xbib.elasticsearch.helper.client.BulkTransportClient][elasticsearch[importer][listener][T#2]] after bulk [202] [succeeded=7102827] [failed=0] [5172ms] [concurrent requests=1]

Can you please tell me how can i increase it or am i missing some other config param