SOLR error: exceeded limit of maxWarmingSearchers=2

[jakubklimek]

When I load a large number of datasets with multiple processes using ckanapi like this: ckanapi load datasets -I /data/ckan.jsonl.gz -z -p 4 -c /etc/ckan/testing/development.ini

Eventually, a Solr error is thrown: ERROR [ckan.model.modification] Solr returned an error: (u'Solr responded with an error (HTTP 503): [Reason: Error opening new searcher. exceeded limit of maxWarmingSearchers=2, try again later.]',)

I could increase the number of WarmingSearchers in Solr or decrease the number of workers for the load, but IMHO the bigger issue here is whether it could be possible for bulk loading to have an option to commit to Solr only after loading all datasets, not one by one, as it still takes a long time this way (9 hours for 130k datasets to load using single worker)

[torfsen]

Similar issue w.r.t. reindexing: ckan/ckan#3482.