for-win icon indicating copy to clipboard operation
for-win copied to clipboard

High memory usage from VMMEM with docker desktop windows x64 V4.12.0

Open vierlijner opened this issue 3 years ago • 86 comments

  • [x] I have tried with the latest version of Docker Desktop
  • [x] I have tried disabling enabled experimental features
  • [x] I have uploaded Diagnostics
  • Diagnostics ID: 1CA50CD6-DD0E-4459-82A8-A1DB31A295F0/20220902065044

Actual behavior

VMMEM has a very high memory usage (2gb) after docker desktop for windows is starting up. And it don't releasing it.

Expected behavior

A low VMMEM usages.

Information

  • Is it reproducible? Yes, install the latest docker v4.12.0 with WSL2.
  • Is the problem new? Yes, it's not like #3858 or #12266 or #1772
  • Did the problem appear with an update? Yes, so I first delete oracle box and vmware, but this wasn't helping. Then i stopped docker and it stopped. So I removed version V4.11.1 (strange because it was the latest) and download the latest V4.12.0. So on deinstalling it detaled all images and containers, so nothing is running.
  • Windows Version: win 10 pro 21H2
  • Docker Desktop Version: v4.12.0
  • WSL2 or Hyper-V backend? WSL2
  • Are you running inside a virtualized Windows e.g. on a cloud server or a VM: No, just on a HP prodesk 400 machine i7-9700 16GB x64

Output of & "C:\Program Files\Docker\Docker\resources\com.docker.diagnose.exe" check

When I start this it doesn't appear.

Steps to reproduce the behavior

  1. Just install the new version v4.12.0 on a win 10 pro 21H2 with wsl2 enabled and the rest default.
  2. Restart PC/ Windows
  3. Watch the VMMEM proces on your machine and see the memory usage. image

Edit: I installed older versions of docker (untill V4.7.1), but all had the same result. So it's not about the version of docker. It's in the windows update from vmmem (or something else) in combine with docker desktop. I'll hope you can notify Microsoft about this (hope more users have this issue) and Microsoft can fix this.

vierlijner avatar Sep 02 '22 07:09 vierlijner

Microsoft has too fix this (with help of docker?)

vierlijner avatar Sep 02 '22 08:09 vierlijner

Same here.

  • Windows Version: win 11 pro 22H2

rventurelli avatar Sep 12 '22 10:09 rventurelli

Same here.

Windows Version: win 11 pro 22H2 ( just updated today )

craigrmann avatar Oct 06 '22 22:10 craigrmann

Same

image

duyanh14 avatar Oct 10 '22 04:10 duyanh14

On fresh installed Windows 10 (21H2 - 19044.2130), Docker impossible to use - it crashes because of 100% Ram usage. Previous Windows 10 version was older, and I had no problems at all.

larinius avatar Oct 13 '22 18:10 larinius

Same here. Usually takes a while, 1 day or so. After the next wake up of hibernation, I run into this problem and the containers do not respond anymore. Killing Docker and waiting for a while (~5 min) does solve the issue, but is annoying.

HerrDerb avatar Nov 08 '22 07:11 HerrDerb

Same here with just 9 services running my PC goes down I can not do anything I should use wsl --shutdown to kill all containers Capture Capture1 Capture3

pouyaSamie avatar Nov 30 '22 12:11 pouyaSamie

Mine is same problem, how to solve this ?

image

image

image

Docker Desktop 4.16.2 (95914)

bomb-kst avatar Jan 27 '23 01:01 bomb-kst

Simply set up memory limit for WSL2:

  1. close docker desktop
  2. execute wsl --shutdown
  3. add following to %UserProfile%\.wslconfig file (create one if there is none):
[wsl2]
memory=2GB
  1. restart docker desktop

Of course you can change 2GB to whatever you like.

fireton avatar Feb 06 '23 13:02 fireton

Simply set up memory limit for WSL2:

  1. close docker desktop
  2. execute wsl --shutdown
  3. add following to %UserProfile%\.wslconfig file (create one if there is none):
[wsl2]
memory=2GB
  1. restart docker desktop

Of course you can change 2GB to whatever you like.

@fireton I've been intestigated and tried to do by this way with 4GB configuration but it still takes me more than 5GB of memory

cuongnvicts avatar Mar 24 '23 03:03 cuongnvicts

Simply set up memory limit for WSL2:

  1. close docker desktop
  2. execute wsl --shutdown
  3. add following to %UserProfile%\.wslconfig file (create one if there is none):
[wsl2]
memory=2GB
  1. restart docker desktop

Of course you can change 2GB to whatever you like.

Setting memory limit will make WSL write to swap which writes a lot of GBs to the disk and by time it will reduce the lifespan of your hard drive. I suppose this should be a temporary workaround till Docker devs figure out why is it consuming so much RAM compared to Linux and MacOS and fix this issue.

ghost avatar Apr 23 '23 17:04 ghost

Facing a similar issue on Windows 10 Home. After stopping Docker Desktop the vmmem does not release memory.

image

Strangely, when working on WSL2 projects on Jetbrains IDEs similar issue occurs after sleeping and waking the laptop up. Looks like there is some Windows related bug as well here.

qwertynik avatar Apr 28 '23 10:04 qwertynik

I have also experienced the same, either an update in Docker or Windows or both. Seems like it might be doing an indexing for search or similar. After a reboot the memory usage basically climbs up to total disk usage on the WSL side. An easy way to decrease the memory usage for me is to docker system prune -a which removes the containers, after that the vmmem memory usage drops to a few gigabytes which I think corresponds quite well to the WSL disk usage. Before I prune the images the memory usage is several gigabytes, up to a total of 32GB (half of my system mem) if I have a lot of containers/images.

At the same time I noticed this change I also noticed that the system works a lot more (CPU fan spinning heavily), when I leave the PC idle for a few minutes. This was not the case before this change.

mnordstr avatar Apr 28 '23 11:04 mnordstr

I suggest to the docker team that if there is a feature that causes this high memory usage please add the option to disable it. I only use docker to run containers and I rarely use the desktop interface.

ghost avatar Apr 28 '23 11:04 ghost

Facing a similar issue on Windows 10 Home. After stopping Docker Desktop the vmmem does not release memory.

image

Strangely, when working on WSL2 projects on Jetbrains IDEs similar issue occurs after sleeping and waking the laptop up. Looks like there is some Windows related bug as well here.

So, from my brief research today it looks like Vmmem is the virtual machine memory pool controlled by Windows and represents all RAM provided to WSL to run all linux kernels. Docker simply uses WSL like a service and is assigned memory from this pool, but WSL could also be running other things, such as a user downloaded Linux distro, so when you quit Docker it doesn't automatically exit WSL.

You can however do this manually with the following command (as mentioned by @fireton):

wsl --shutdown

This will free up all memory used by Vmmem.

geoh avatar Apr 30 '23 11:04 geoh

@geoh What is the point of running wsl --shutdown and rerunning docker? It still has high memory usage on startup and without any container running.

ghost avatar Apr 30 '23 11:04 ghost

Sure @RouisIbra, Docker has high memory usage and this doesn't fix that. But, if you have pivoted tasks and no longer need to run Docker, this will free up that memory that is still being used even after Docker has been closed.

Also, if heavy usage of Docker has pushed that Vmmem memory pool to a significant size and you have now stopped those Docker containers, then closing docker, stopping Vmmem and restarting Docker will reset the memory usage back to initial levels... so this might help in certain situations.

geoh avatar Apr 30 '23 12:04 geoh

I had the same issue with Windows 10 Version 20H2 and OS Build 19042.1466 and creating the ".wslconfig" file as the guys told solved my problem. But my concern: is this solution like a memory suppression for the containers and made my container slow? Is there any idea regarding this memory usage? Is it a memory leakage? Or is it only memory reservation that Docker/WSL2 doing? I didn't face this issue in Windows 11 Version 22H2 and OS build 22621.1555 till now at least.

sia72abedi avatar May 05 '23 09:05 sia72abedi

The same happens for Windows Server 2022 datacenter edition.

geralexgr avatar May 08 '23 13:05 geralexgr

Outrageous background memory usage immediately on startup, even when no containers are active. Noticed in the last week or two. Adding a .wslconfig file seems like a temporary solution at best, there's no reason to be using that much memory when containers aren't even running.

Windows 10 Enterprise Docker Desktop 4.19.0 (106363)

dalexander-trc avatar May 08 '23 14:05 dalexander-trc

Same issue. Whether from a fresh reboot or logout/login, then launching Docker Desktop without any containers active at all, it jumps from about 700MB used to:

image

This is destroying my ability to get work done. I'll try the .wslconfig workaround, but it seems like a band-aid on a broken limb.

Windows 11, 21H2, Build 22000.1817 Docker Desktop for Windows 4.19.0

SturmB avatar May 10 '23 00:05 SturmB

No containers running, vmmem limited to 8 GB: image (Containers use 0 GB of RAM, Docker uses 3.4 GB while idle)

$ wsl -d docker-desktop -> top image

containerd and dockerd are still using ~2 GB in my case while everything is technically idle...

Falco20019 avatar May 11 '23 12:05 Falco20019

Same here, 20GB of Vmmem, draining all my resources.

MattiaGallegati avatar May 14 '23 07:05 MattiaGallegati

Facing a similar issue on Windows 10 Home. After stopping Docker Desktop the vmmem does not release memory. image Strangely, when working on WSL2 projects on Jetbrains IDEs similar issue occurs after sleeping and waking the laptop up. Looks like there is some Windows related bug as well here.

So, from my brief research today it looks like Vmmem is the virtual machine memory pool controlled by Windows and represents all RAM provided to WSL to run all linux kernels. Docker simply uses WSL like a service and is assigned memory from this pool, but WSL could also be running other things, such as a user downloaded Linux distro, so when you quit Docker it doesn't automatically exit WSL.

You can however do this manually with the following command (as mentioned by @fireton):

wsl --shutdown

This will free up all memory used by Vmmem.

Thanks for sharing your research insights. Yes, I am aware of the fact that vmmem would include all the virtualization related processes. In this case, nothing besides WSL and Docker Desktop was running at least from what I know by keeping track of which apps were started.

qwertynik avatar May 27 '23 11:05 qwertynik

vmmem usage when nothing is running: CPU - 0.5-2%, RAM - < 900MB RAM vmmem usage after just starting Docker Desktop on Windows: CPU - <10%, RAM - < ~5,000MB RAM.

After closing Docker Desktop vmmem's RAM usage remains high. Restarting wsl with wsl --shutdown brings things back to normal.

Would appreciate it if the Docker team can look into this and suggest solutions.

qwertynik avatar May 27 '23 11:05 qwertynik

This is not docker specific. WSL will by default take up to half your system memory as memory. However, pages marked as free will not be allocated. But page and inode caches will be. So, any filesystem activity will dirty pages that get allocated, and these pages are never freed by themselves.

However, there is a command to drop all inode and page caches:

# echo 3 > /proc/sys/vm/drop_caches

This command is non-destructive and no data will be lost. However, as nothing is cached after the command, performance may be somewhat affected until everything is cached.

For a bit more savings, you can add a sync first to write out all dirty pages. Note that sync does not wait for the pages to get written, so waiting may be a good idea:

# sync; sleep 5; echo 3 > /proc/sys/vm/drop_caches

Docker immediately shows the reduced size. VmmemWSL takes a while to shrink.

Further reading: https://www.tecmint.com/clear-ram-memory-cache-buffer-and-swap-space-on-linux/ https://www.kernel.org/doc/Documentation/sysctl/vm.txt

Edit: This seems to get fixed by Microsoft in WSL 1.3.10+, see https://github.com/microsoft/WSL/releases

clh38626 avatar Jun 15 '23 12:06 clh38626

Version 1.3.10 doesn't fix it. wsl --version WSL version: 1.3.10.0 Kernel version: 5.15.90.2-3 WSLg version: 1.0.54 MSRDC version: 1.2.4240 Direct3D version: 1.608.2-61064218 DXCore version: 10.0.25880.1000-230602-1350.main Windows version: 10.0.19045.3086 My containers eat as much memory as I can give. Docker Desktop "Resource Usage" plugin shows, that overall memory consumption is 2.67 GB. vmmem memory usage slowly (several minutes) grows up to 14 GB (current max limit in wslconfig). "Page faults" counter for vmmem process increases rapidly.

About three weeks ago all was fine and vmmem actual memory usage was similar to value in "Resource Usage". :(

UPD: So as 1.3.11 version....

ilya-girman avatar Jun 21 '23 08:06 ilya-girman

Same here. It eats my laptops memory until it becomes unresponsive.

the-gozo avatar Jun 21 '23 12:06 the-gozo

With 1.3.11 it works for me, if WSL is truly idle. It fails, if Docker Desktop is running or any other process (even top) is running.

clh38626 avatar Jun 21 '23 20:06 clh38626

I have this same issue image no containers open, wsl2 is idle

atticusrussell avatar Jun 23 '23 02:06 atticusrussell