mdox icon indicating copy to clipboard operation
mdox copied to clipboard

Add concurrency to `mdformatter`

Open saswatamcode opened this issue 4 years ago • 2 comments

This PR parallelizes the file processing loop in mdformatter for faster formatting and link checking using WaitGroup by spawning a goroutine for each file. (This also changes CLI spinner behavior as filenames cannot be shown due to multiple files being processed at once)

Some caveats,

  • Panic during cancellation sometimes due to v.c.Wait()(flaky, maybe due to internal colly WaitGroup)
  • Need some way to limit number of file processing goroutines

saswatamcode avatar Jul 23 '21 13:07 saswatamcode

Still comments not addressed, I believe?

bwplotka avatar May 10 '22 20:05 bwplotka

Any progress?

bwplotka avatar Jun 07 '23 08:06 bwplotka