Tailwind CLI slow down / memory leak

[FallDownTheSystem]

What version of Tailwind CSS are you using?

v3.0.22

What build tool (or framework if it abstracts the build tool) are you using?

None

What version of Node.js are you using?

v17.0.1

What browser are you using?

N/A

What operating system are you using?

Windows 10

Reproduction URL

https://github.com/FallDownTheSystem/tailwind-cli-slowdown

Describe your issue

After saving a file in the root folder, that triggers a rebuild by the Tailwind CLI watcher, while the rebuild is still in progress, I think some kind of memory leak happens.

The reproduction requires a file to be saved very rapidly to showcase it, but on larger projects, it can happen naturally, as the build times are longer to begin with.

I'll paste the reproduction steps and explanation I added to the README.md of the minimal reproduction demo here. I've also attached a video that showcases the behavior.

https://github.com/FallDownTheSystem/tailwind-cli-slowdown

1. npm install
2. npm run watch
3. Spam save ./folder/noonroot.aspx or ./folder/nonroot2.aspx (On Windows you can hold down ctrl + save to rapidly save the file)
4. Spam save ./root.aspx for a long while
5. Try to spam save one of the nonroot.aspx files again

The CLI now gets "stuck" on adding the rebuild step to the promise chain faster than it can process then, making the chain longer and longer. Once you stop spamming save, the chain will unwind and all the rebuilds will complete. But now each time you attempt to save, the process allocates a larger chunk of memory than originally.

This is even more evident if you spam save the tailwind.config.js file. This takes even longer, and seems to reserve much more memory.

After a while, the memory will be released, but subsequent saves of the noonroot.aspx files will cause much larger chunks of memory to be allocated, and the built times have increased by an order of magnitude.

At the extreme, this will lead to an out-of-memory exception and the node process will crash.

This bug seems to only happen when you edit one of the files in the root folder, and is more evident on larger projects where the building times are longer to begin with, and thus the memory 'leak' becomes more apparent faster.

This is harder to reproduce, but from experience, I would argue that this memory 'leak' happens often when you attempt to save a file while the rebuild is still in process. In a larger project, my watcher node process will crash several times a day due to out-of-memory exceptions.

The repository also includes a modified-cli.js that I used for debugging purposes. The modified Tailwind CLI has the addition of logging when the watcher runs the on change handler, and when the promise chain is increased or decreased.

https://user-images.githubusercontent.com/8807171/153777233-54acb464-d31f-4cab-8163-5f035060b85a.mp4

What cannot be seen on the video is the memory usage, which at its peak got up to 4 GB.

[FallDownTheSystem]

EDIT: I think it has something to with the content glob matching, because it doesn't seem to slow down with the following tailwind.config.js

module.exports = {
	content: ['./root.aspx', './folder/nonroot.aspx', './folder/nonroot2.aspx']
};

[thecrypticace]

I can reproduce the memory usage going up. However, it seems to go back down fairly quickly. I'm not certain that this is something we have any control over (directly or indirectly). I'll make a note to investigate this further to see what we can do here.

Regarding your comment about the glob:

This may be because your glob pattern caused it to scan node_modules looking for .aspx files. Depending on your disk speed, the size of node_modules, computer load, etc… this could be 100ms+ of work just looking for files. This could be contributing to the chain issue but I'm not certain. Given that you discovered you're not seeing issues without the root glob I suggest naming any files in the root and then using glob for only the folders you know you need to scan. That would look something like this:

module.exports = {
    content: ['./root.aspx', './folder/**/*.aspx']
};

[FallDownTheSystem]

Thanks for looking into this. I think not scanning the node_modules folder is solid advice 😄

And yes, the memory usage does seem to go back down, but not all the way. I just tested again and after spamming a while it stays around 300-400 MB, and a single save after that spikes the memory usage to 1.6GB, and the build takes 750ms, though I'd argue this is actually longer, 750ms is only for the measured part. For reference, originally the node process takes 50-70 MB and a build takes 30ms.

Here's a video of a single save after having spammed the saves for a while. You can see it start at 400 MB, spikes to 1.6 GB but after it comes down it goes to 460MB and stays there.

https://user-images.githubusercontent.com/8807171/153954099-4c56026f-123d-4bb1-a0b4-66b915588a9d.mp4

I also think (but am not sure) that these kinds of memory leaks or degradations in performance can happen even when you don't do the ridiculous spamming, or have it scan folders with tens of thousands of files.

I wasn't able to figure out where the memory usage is actually increasing, it's possible it's in one of the libraries the CLI calls.

[FallDownTheSystem]

Also clearly this is not a widespread issue and is mostly mitigated by not doing anything too dumb, like scanning node_modules, so feel free to close to the issue if you feel it's not worth pursuing.

[thecrypticace]

I think it's definitely still worth at least looking into but given that it's not more widespread it's lower priority. I tested a setup where I saved the file ~10,000 times and the process never grew by more than ~20MB-ish in overall usage after it went back down. Might be worth me just letting something run over and over for 15–30m in the background to see what happens.

[FallDownTheSystem]

I was testing this by spam saving the tailwind.config.js itself, which produces the same result basically.

One thing I noticed is that these lines:

https://github.com/tailwindlabs/tailwindcss/blob/db475be6ddf087ff96cc326e891ac125d4f9e4e8/src/cli.js#L763-L772

should probably be inside the chain callback. Now they're being called every time the watcher runs (when editing a context dependency). Having made that change to the CLI, it mitigated the issue further, and the memory usage seems to drop down to the normals levels < 100 MB, even after peaking at above 1GB. Subsequent saves are also fast, but spam saving becomes immediately slow still, although the slowness recovers much faster after I stop spamming saves, so something's still afoot.

I'm not sure what, but either some other code runs, besides the watcher's on change handler, or there's something async that's not being awaited, or the method of chaining promises by chain = chain.then(... is not safe.

[FallDownTheSystem]

The behavior still happens, even if the watcher event handlers are empty, so this seems to be an issue with chokidar. That said, moving the lines I mentioned above inside the chain is still probably a good idea.

[jbergstroem]

I use tailwind in a SvelteKit setup and noticed a similar memory leak. After dumping the heap and looking at it, you can see a lot of older instances of the compilation still hanging around (I ran global.gc() before to make sure). MacOS 12.3 (Intel) / node 16.14.1 and tailwindcss 3.0.23:

[willdean]

We also run the CLI in watch mode on Windows and notice it occasionally running out of memory and crashing Node. It's infrequent enough that I haven't bothered to report it previously, so perhaps the problem is more widespread than might be inferred from the number of GH issues.

We also see a gradual increase in duplicate content output to the .css file over many compiles which we clean-up by stopping the CLI and restarting it - this forces a clean build. Clearly there is some kind of state that can hang-around in the CLI between compiles in some circumstances. Unfortunately it's completely impractical for me to provide a repro URL for this.

[iMuFeng]

We are facing the same issue, memory leaks once add something to the content option.

[philiparvidsson]

Same problem here. I don't think I'm doing anything dumb. My project isn't even particularly big, but after about 20 minutes of saving files, tailwindcss slows down to a crawl.

[geraldodev]

I keep updating tailwind to css if it solves this problem. After a while It generates a message of out of memory error. I'm not starting it anymore when I know that I'm not going to use it, which is not ideal.

[FallDownTheSystem]

I was testing this by spam saving the tailwind.config.js itself, which produces the same result basically.

One thing I noticed is that these lines:

https://github.com/tailwindlabs/tailwindcss/blob/db475be6ddf087ff96cc326e891ac125d4f9e4e8/src/cli.js#L763-L772

should probably be inside the chain callback. Now they're being called every time the watcher runs (when editing a context dependency). Having made that change to the CLI, it mitigated the issue further, and the memory usage seems to drop down to the normals levels < 100 MB, even after peaking at above 1GB. Subsequent saves are also fast, but spam saving becomes immediately slow still, although the slowness recovers much faster after I stop spamming saves, so something's still afoot.

I'm not sure what, but either some other code runs, besides the watcher's on change handler, or there's something async that's not being awaited, or the method of chaining promises by chain = chain.then(... is not safe.

I still think this would at least alleviate the problem. The refreshConfig and all the rest should be called within the callback, so that they're properly initialized when the actual rebuild happens. Most of the time this is a nonissue, but when things start slowing down, these can get out of sync.

[thecrypticace]

Turns out we were essentially doubling the rule cache (not quite but close enough) instead of just adding the few entries to it that needed to be added. This can result in a significant slow down in fairly specific situations.

I'm hoping this has fixed a good portion of the issue here. Can some of you please give our insiders build a test and see if it helps out at all? I'm hopeful it'll have some positive impact but if it isn't sufficient I'll reopen this issue.

Thanks for all the help and bearing with us on this one. ✨