SolrTextTagger icon indicating copy to clipboard operation
SolrTextTagger copied to clipboard

Posting gzip text file to improve performance

Open ewianda opened this issue 8 years ago • 2 comments
trafficstars

Hi David First off thanks for the amazing software. I am using this to extract tokens from 2 Million plus text files of 200K average size. I was looking at compressing the files to improve bandwidth usage. I followed Gzip Filter for jetty configuration with no success. I went as far as getting the following error. TaggerRequestHandler requires text to be POSTed to it. I was wondering if zip files could be decompressed within this handler. Unfortunately I am not a proficient java programmer to attempt such changes.

ewianda avatar Feb 06 '17 00:02 ewianda

There's some info here: http://signaldump.org/solr/qpod/33137/applying-gzip-compression-in-solr-5-1

dsmiley avatar Feb 06 '17 18:02 dsmiley

Thanks for the quick response. I actually looked at this information but was wary about the No .war changes in version Solr 5.3 and later and I am using Solr 5.4.1 UPDATE If I apply the configurations suggested in GzipFilter configuration and set my http headers to {'content-type':'text/plain','transfer-encoding': 'gzip','accept-encoding':'gzip,deflate'} I don't get the error TaggerRequestHandler requires text to be POSTed to it. but no tags are return if I my post data is gzip compressed.

ewianda avatar Feb 06 '17 22:02 ewianda