frog icon indicating copy to clipboard operation
frog copied to clipboard

Resize images asynchronously

Open gerdint opened this issue 7 years ago • 2 comments

Yields a 2x speed-up on my dual-core i5 machine.

However, as it stands each size of an image to be resized yields a new subprocess potentially spawning many subprocesses concurrently if the site contains a lot of images (such as my own). This increases memory usage and may therefore cause disk swapping if memory is limited. Does not seem to be a big issue on my 8GB SSD-equipped machine but I understand performance characteristics may be different on other machines.

It may perhaps be preferable to a have a job server making sure that no more than N processes are spawned concurrently. A useful but potentially conservative value for N may be the number of processor cores.

OR perhaps I'm over-engineering here. WDYT?

gerdint avatar Oct 31 '16 22:10 gerdint

I went ahead and implemented a simple process pool. Really helped to lower memory usage while retaining the faster build times (on multi-core machines).

This was my first foray into concurrent Racket programming. As usual Racket seems to excel in this area and while things felt intuitive and I tried to keep things simple you may want to review the last commit more carefully.

Personally I feel pretty happy with the state of things now and barring any bugs I do not see any need for more work on this feature. The srcset-async branch is only two commits on top of the srcset one so once the latter is squashed it should only be a matter of a trivial rebase onto that for merging it.

gerdint avatar Nov 09 '16 22:11 gerdint

I have spotted one issue when using this together with the Frog watch mode. Will look into it.

gerdint avatar Jan 11 '17 21:01 gerdint