grunt-s3
grunt-s3 copied to clipboard
pifantastic
Error: socket hang up
I sometimes get socket errors when trying to push content to s3:
Why does this happen? It seems to be non-deterministic.
As a hacky workaround, I've broken it up into smaller sets of files on each task. That seems to solve the problem.
function makeSubTasks(dir) {
grunt.file.expand(path.join(dir, '*')).forEach(function(dir) {
grunt.config('s3.' + dir + '.upload', uploadsOfDir(dir));
});
}
makeSubTasks(devRoot);
// it's kinda shitty to have to do this on grunt.initConfig(), because
// scanning all the files is slow and putting it there means that
// we will do it every time the user types `grunt`
function uploadsOfDir(dir) {
var files = grunt.file.expand(path.join(dir, '**', '*.*'));
return files.map(function(file) {
return {
src: file,
dest: path.sep + file.split(path.sep).slice(1).join(path.sep)
}
});
}
Yeah, I've witnessed this too. I imagine it's just s3 being finicky. I've considered adding automatic retry to everything.
Interesting. Is it possibly related to knox? s3 is supposed to be able to handle huge uploads, right? How can it be failing for < 500 files?
Automatic retrying sounds reasonable.
hmm, good point about knox. There is this issue: https://github.com/LearnBoost/knox/issues/116
Have you tried grunt-s3 0.2.0-alpha.2? It uses a version of knox that has the fix from that issue.
I have been using that version of grunt-s3, and I'm still getting the same issue.
I can repeat the problem with grunt-s3 0.2.0-alpha.2 and Node 0.10.10. It doesn't happen all the time, just sometimes.
This issue is happening constantly for me as well with a large amount of files.
+1 Had not seen this error before a month ago, now it's happening on every upload.
+1 happens intermittently but frequently with grunt-s3 0.2.0-alpha.2 and Node 0.10.11
I think (theoretically) this is due to not calling .end() on the knox calls. I'm playing with fixing this too.
Any updates/fixes for this one? It's happening consistently on uploading to S3 for me.
+1 this intermittent error happens with node v0.10.22 and grunt [email protected], [email protected] (on mac os x maveriks).
