ftptree

ftptree copied to clipboard

FTP concurrent requests problem

1

Hi. I have put CONCURRENT_REQUESTS = 1 CONCURRENT_REQUESTS_PER_IP = 1 CONCURRENT_REQUESTS_PER_DOMAIN = 1 CONCURRENT_ITEMS = 1 in ftptree_crawler/settings.py Anyway, soon after starting crawler, I get an error: ConnectionLost: ('FTP connection...

draganpodvezanec

when test ftptree at command line: ./crawltree.py sites/ucsc.json it responses: File "./crawltree.py", line 231, in with open(config['tree_file'], 'w') as op: IOError: [Errno 2] No such file or directory: u'trees/ucsc.json' where...

0s1nj3ct0r

Without a license for the repo, it's unclear under what conditions, if any, other people are allowed to reuse this code.

icopp

Bit of a hack to fix this in the short term https://github.com/hammer/ftptree/commit/f5df59321f2c1b689b5bea50cca1f5be421f5594

hammer

Consider using ftputil

2

http://ftputil.sschwarzer.net/trac

hammer