hbeanstalk icon indicating copy to clipboard operation
hbeanstalk copied to clipboard

listTubes fails with large number of tubes

Open scsibug opened this issue 14 years ago • 3 comments

Test case "ListTubes" fails unpredictably when the number of tubes gets large (observed with ~7000 tubes).

Error in: 19:ListTubes

user error (syntax error: line -114, column 1) Cases: 29 Tried: 29 Errors: 1 Failures: 0 Counts {cases = 29, tried = 29, errors = 1, failures = 0}

scsibug avatar Jun 11 '10 23:06 scsibug

Source of error is probably in Data.Yaml.Syck

scsibug avatar Jun 11 '10 23:06 scsibug

When creating lots of tubes, observed error (from beanstalkd): beanstalkd: prot.c:709 in check_err: writev(): Broken pipe

scsibug avatar Jun 12 '10 00:06 scsibug

Data.Yaml.Syck is not the source of the problem; I've reproduced it with my own parser. The problem is that, when the tube list is very long, a single call to recv does not read the whole reply. You need to repeatedly call recv to build up the reply, and then parse it.

Alternately, you could look at my fork of the code, where I've switched over to bytestrings and attoparsec, which solves this bug with incremental parsing.

PeterScott avatar Jan 26 '11 00:01 PeterScott