yarnpkg
tmp directory filled by yarn (and not cleaned)
Do you want to request a feature or report a bug? bug
What is the current behavior? Each "yarn run" will create a directory under "/tmp" directory which looks like
/tmp/yarn--1542302743598-0.6454897498245407
If the current behavior is a bug, please provide the steps to reproduce. Let's say we have a script in package.json as follow:
"scripts": {
"dummy": "echo 'dummy'"
}
If we play this dummy script and we check number of directories before and after in /tmp:
$> ls -dla /tmp/yarn* | grep $USER | wc -l
0
$> yarn run dummy
yarn run v1.12.3
$ echo 'dummy'
dummy
Done in 0.12s.
$> ls -dla /tmp/yarn* | grep $USER | wc -l
1
Running the dummy script will increment the result each time. FYI:
- yarn is installed globally
- bug seems to appear somewhere between v1.10.0 and v1.11.0 (I can't reproduce with 1.10.0)
What is the expected behavior? Tmp directory should not be filled indefinitely: it should reuse the already created temporary files (if tmp is really needed) or it should have some kind of cleaning mechanism to avoid overwhelmingly the tmp directory.
Please mention your node.js, yarn and operating system version. node: v8.9.4 nvm: 0.33.2 yarn: 1.12.3 os: Debian 3.16.51-3
I am getting the same results on 1.12.3. Not really sure why this is happening?
I noticed each tmp entry has a shell script relating to node and yarn. I have tried running yarn run with different flags such as --link-folder, --global-folder and --cache-folder but I can't seem to prevent the directories from being created.
Yarn creates a temporary directory each time a script is executed, in order to setup "fake" Node and "fake" Yarn binaries that point towards the exact same Yarn/Node than those currently being executed (to prevent the case where the global Node is different from the one used to run Yarn itself).
The problem is that they never get removed at the moment: https://github.com/yarnpkg/yarn/blob/master/src/util/execute-lifecycle-script.js#L31-L48
My /tmp is 24gb almost all of which is the Yarn files described above.
me@me:~$ sudo du -hs /tmp
24G /tmp
me@me:~$ uptime
15:58:04 up 52 days, 5:30, 1 user, load average: 0.97, 0.87, 0.50
Was this a regression? I don't remember my CI agents being filled by this in the past.
In the meantime.... Is it safe to manually remove all of them? Or the removal could break anything?
I wrote crontab that does "mkdir -p /tmp/yarn & mv /tmp/yarn-* /tmp/yarn/" (typing by memory, don't have my laptop to check). In Linux if you rename directory it still can be used by program that already has file descriptor, same goes for deletion I think. Never had issues with renaming via cron, never had issues removing them manually, I would assume it's probably safe to delete it from cron as well. Worst case scenario I would assume your currently running "yarn run something" will crash.
On Mon, Jun 17, 2019, 12:33 Xavi Montero [email protected] wrote:
In the meantime.... Is it sure to manually remove all of them? Or removal could break anything?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/yarnpkg/yarn/issues/6685?email_source=notifications&email_token=AAABW4UKLTC7IRND2RCFOTDP27RIZA5CNFSM4GEERUT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX4G3BQ#issuecomment-502820230, or mute the thread https://github.com/notifications/unsubscribe-auth/AAABW4SVFYLVBE4F3H7ZRQ3P27RIZANCNFSM4GEERUTQ .
@xmontero We removed files older than 1 day on our dev server.
Never had issue since then.
10 4 * * * find /tmp/ -name "yarn*" -type d -mtime +1 -exec rm -rf {} \; > /dev/null
There is no risk removing those files when Yarn isn't running. Even when running, it should be relatively safe as they're usually only used when the program boots.
Fwiw this problem will be fixed from the v2 onward; the artifacts will be properly discarded right after the commands return.
Running build in Docker image is also a good workaround. It protects you from such leaks to CI workers system, by design.
On Tue, Jun 18, 2019, 11:37 Maël Nison [email protected] wrote:
There is no risk removing those files when Yarn isn't running. Even when running, it should be relatively safe as they're usually only used when the program boots.
Fwiw this problem will be fixed from the v2 onward; the artifacts will be properly discarded right after the commands return.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/yarnpkg/yarn/issues/6685?email_source=notifications&email_token=AADISDU4FFFFWZZI37RRY4TP3CUGXA5CNFSM4GEERUT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX5ZV4I#issuecomment-503028465, or mute the thread https://github.com/notifications/unsubscribe-auth/AADISDVU55SPQE54EYNMLVDP3CUGXANCNFSM4GEERUTQ .
Is this issue persistent with yarn v2? If anyone has tried this, please let me know. I am giving it a shot today with v2 on one of our apps to check this in the mean time.
De issue still exists in the latest version
sh yarn -version
1.22.4
sh ls -alF
drwxr-xr-x 2 username username 4096 Jun 9 11:57 yarn--1591696641892-0.05204308679429248/ drwxr-xr-x 2 username username 4096 Jun 10 14:55 yarn--1591793724850-0.4196722332063454/ drwxr-xr-x 2 username username 4096 Jun 10 15:15 yarn--1591794953501-0.3040314059514666/ drwxr-xr-x 2 username username 4096 Jun 10 15:26 yarn--1591795571216-0.8222359319740244/ drwxr-xr-x 2 username username 4096 Jun 11 09:33 yarn--1591860782625-0.15962323877699136/ drwxr-xr-x 2 username username 4096 Jun 11 09:36 yarn--1591860987612-0.0746955607161679/ drwxr-xr-x 2 username username 4096 Jun 11 11:03 yarn--1591866191467-0.6557044172705777/ drwxr-xr-x 2 username username 4096 Jun 11 11:04 yarn--1591866263355-0.7429486092878892/ drwxr-xr-x 2 username username 4096 Jun 11 11:07 yarn--1591866427043-0.3962468811805635/ drwxr-xr-x 2 username username 4096 Jun 11 12:50 yarn--1591872605318-0.8161204330414475/ drwxr-xr-x 2 username username 4096 Jun 11 12:50 yarn--1591872605322-0.48513192538032124/ drwxr-xr-x 2 username username 4096 Jun 11 12:50 yarn--1591872622244-0.4583333405082548/ drwxr-xr-x 2 username username 4096 Jun 11 13:59 yarn--1591876762200-0.05283671445181093/ drwxr-xr-x 2 username username 4096 Jun 11 14:09 yarn--1591877355444-0.7000002370403433/
I can confirm this still happens on the very same version (1.22.4).
Platform: Debian 10 on WSL - Windows 10 x64 ver 19041 (2004).
Any update on this? we still have several problems with hard disk and "no left space" on Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-38-generic x86_64) with
yarn -v
1.13.0
Yarn was so good, until this. Funny thing is this issue has been reported since like more than 2 years ago, and no collaborator care about fixing this, even there is an existing PR #7207 to review.
@arcanis wrote:
Fwiw this problem will be fixed from the v2 onward; the artifacts will be properly discarded right after the commands return.
Just for clarification (even if it will hurt), I guess this means version 1.x will not receive a fix, but 2.x has the issue fixed? In this case the issue could IMO be closed, but it would be nice to have clarity on that.
Comparing to the description of this repository itself:
The 1.x line is frozen - features and bugfixes now happen on https://github.com/yarnpkg/berry
Just for clarification (even if it will hurt), I guess this means version 1.x will not receive a fix, but 2.x has the issue fixed? In this case the issue could IMO be closed, but it would be nice to have clarity on that.
Indeed. Note that the migration should be fairly painless if you follow the migration guide, since it explains how to stay on the typical node_modules install strategy even with 2+.
Please fix your stuff. This bug is 3 years old, and it is probably only a single line of code that deletes the temporary files. Filling the tmp/ directory until it's full is a major defect, and I hate being forced to upgrade to a newer version just because of this.
Is there a "manual" script we could run that would clean up after yarn finished?
Is there a "manual" script we could run that would clean up after yarn finished?
On Unix-based systems (Linux & co):
rm /tmp/yarn--*
Is there a "manual" script we could run that would clean up after yarn finished?
on Windows OS this can be used:
for /d /r "%temp%" %i in (yarn--*) do @rmdir /s /q "%i"
I clear tmp directory in every hour by crontab, but sometimes server dies because of this(even at dawn!!) I cannot figure out why my server access to the tmp file at 11:28(not an exact 11:00 or 12:00).
But if I do not clear this regulary, our server instance is filled by yarn! I'm in a dilemma.
.
version 1.22.19 still exsits. the problem reported in 2018, and still not fixed in 2023 :)
Got this issue too on a Nuxt.js (Node 16 / Yarn 1.22.19) project hosted on Platform.sh. It created new yarn files at each build/merge!
We had to clear the /tmp (and we are searching for a way to clean it at each build) because it literally ate 8GB of tmp space.
