fscrawler
fscrawler copied to clipboard
dadoonet
Elasticsearch File System Crawler (FS Crawler)
Fscrawler Version: 2.6 Elasicsearch V: 6.4.3 I am not able to crawl files which are larger in size, i did changes in "indexed.chars" in _settngs.json file. is there any other...
I want to add additional tag while indexing local files using fscrawler. Using rest API Additional tags we can achieve requirement but while indexing local file,can we archive this functionality....
The next version of the Low Level client (7.5) can decode the cloud Id (see https://github.com/elastic/elasticsearch/pull/47868). So we can remove our internal stuff.
**Describe the bug** Can't use the loadbalancer URL to access elasticsearch cluster from fscrawler using the elasticsearch nodes setting. **To Reproduce** Steps to reproduce the behavior: Install nginx on kubernetes....
I am still fiddling around to find a way to have fscrawler index multiple folders at once, while writing to the same index on my ES. If anything goes south...
We are considering `fscrawler` so much as our document indexing tools where we are processing more than 32 millions of docs every day (average). But our usecase is, we are...
It would be helpful if we can deploy `fscrawler` with all the necessary configuration as CLI parameters. Originally the idea was started flying from https://github.com/dadoonet/fscrawler/pull/849 this PR. Instead of discussing...
I frequently change data, so my json files inside fscrawler's tmp / es directory are constantly changing. may be deleting, may be changing the data inside a json file, or...
I'm trying to use FS Crawler 2.6 in a Windows Server machine to index a huge number of files in my company. It's a very large Windows folders tree in...
When indexing large documents you may hit limits not only on the indexing part, but also when doing searches. Splitting documents into one entry per page helps slice up large...
