Mathijs de Bruin
Mathijs de Bruin
even SOCKET_READ_TIMEOUT is set, the nanohttpd doesn't interrupt, rather, it will continue. after some time the response is returned and sent back to python which is already closed the connection....
At this moment, TIKA is using lots of memory as there is a parser for every thread. However, it seems that at least AutoDetectParser is thread safe and hence should...
http://www.rationaljava.com/2015/03/creating-web-services-and-rest-server.html Right now, every request gets its own thread which can become expensive with lots of threads. Having a 'proper' webserver might provide for better queue managemnt.
Rebase of @jimmykodes 's `slice-support` branch (#13).
Draft/WIP pull request for: - [x] New index mappings, different indexex for various file types. - [x] Automatic index creation (closes #223 and supersedes #228). - [x] Refactor alias resolution,...
Enable refactored, much cleaner worker pool implementation with load limiter (preventing overloading of nodes e.g. due to user traffic). Ref 7b86a97f5f5672d20e1575bb674e578a29b39969 and 871729421ab2fe25ac133a254eb6d4e7cb643513.
``` Dec 17 17:48:37 lovelace ipfs-crawler[627487]: Done crawling 'ipfs://QmRYNAwxEcTcvYDfTFJZKtUpSQzUgAqToJvgTEnUnLz8mb', result: cache error in writing &{} to QmRYNAwxEcTcvYDfTFJZKtUpSQzUgAqToJvgTEnUnLz8mb: closed Dec 17 17:48:38 lovelace ipfs-crawler[627487]: Done crawling 'ipfs://QmUSYH8tGs2BJyDLu5onzCUVifU7Z494q9UDQKsqKDp1iJ', result: cache error in...
And connect to them before crawling. Both our sniffer as well as our pinservice should give information on where content is available.