esformatter icon indicating copy to clipboard operation
esformatter copied to clipboard

process multiple files in parallel

Open millermedeiros opened this issue 8 years ago • 4 comments

something similar to the way jscodeshift works (create multiple child processes and run transforms in parallel).

this should speed things up drastically when processing large codebases.

PS: this is a very good candidate for a dedicated project, maybe there is something on npm that we can reuse.

millermedeiros avatar Jul 06 '16 03:07 millermedeiros

Fyi, you can use xargs -n 50 -p 8 to run the command on 8 cores using 50 files at a time.

find src -name '*.js' | xargs -n 50 -p 8 esformatter

vjeux avatar Jan 17 '17 06:01 vjeux

see also: https://www.npmjs.com/package/worker-farm

millermedeiros avatar Aug 21 '17 19:08 millermedeiros

@vjeux I'm probably wrong as it's not my best field but I've read "find -exec" is safer than "find + xargs". Feel free to search or just ignore me :P

nmaxcom avatar Aug 29 '17 23:08 nmaxcom

@nmaxcom yeahh, if you have files with quotes or spaces, then you are going to be screwed. But if someone is able to create arbitrary named files on your file system, you are dead anyway, they can just change any js file and run arbitrary code.

vjeux avatar Aug 29 '17 23:08 vjeux