x/tools/gopls: high memory consumption

[codisms]

gopls version

Build info
----------
golang.org/x/tools/gopls v0.9.1
    golang.org/x/tools/[email protected] h1:SigsTL4Hpv3a6b/a7oPCLRv5QUeSM6PZNdta1oKY4B0=
    github.com/google/[email protected] h1:81/ik6ipDQS2aGcBfIN5dHDB36BwrStyeAQquSYCV4o=
    github.com/sergi/[email protected] h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
    golang.org/x/[email protected] h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
    golang.org/x/[email protected] h1:5KslGYwFpkhGh+Q16bwMP3cOontH8FOep7tGV86Y7SQ=
    golang.org/x/[email protected] h1:rm+CHSpPEEW2IsXUib1ThaHIjuBVZjxNgSKmBLFfD4c=
    golang.org/x/[email protected] h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
    golang.org/x/[email protected] h1:Ljlba2fVWOA1049JjsKii44g8nZN2GjpxMlzVc8AnQM=
    mvdan.cc/[email protected] h1:kTojdZo9AcEYbQYhGuLf/zszYthRdhDNDUi2JKTxas4=
    mvdan.cc/xurls/[email protected] h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
go: go1.16.15

go env

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ubuntu/.cache/go-build"
GOENV="/home/ubuntu/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/ubuntu/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ubuntu/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.15"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3575468840=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Editing Go files with vim and vim-go.

What did you expect to see?

Lower memory consumption.

What did you see instead?

Over 5GB of memory consumption; thrashing swap file.

Editor and settings

VIM - Vi IMproved 9.0 (2022 Jun 28, compiled Aug  1 2022 05:17:14)
Included patches: 1-124
Compiled by ubuntu@dev-veritone
Huge version without GUI.  Features included (+) or not (-):
+acl               +file_in_path      +mouse_urxvt       -tag_any_white
+arabic            +find_in_path      +mouse_xterm       -tcl
+autocmd           +float             +multi_byte        +termguicolors
+autochdir         +folding           +multi_lang        +terminal
-autoservername    -footer            -mzscheme          +terminfo
-balloon_eval      +fork()            +netbeans_intg     +termresponse
+balloon_eval_term +gettext           +num64             +textobjects
-browse            -hangul_input      +packages          +textprop
++builtin_terms    +iconv             +path_extra        +timers
+byte_offset       +insert_expand     +perl/dyn          +title
+channel           +ipv6              +persistent_undo   -toolbar
+cindent           +job               +popupwin          +user_commands
-clientserver      +jumplist          +postscript        +vartabs
-clipboard         +keymap            +printer           +vertsplit
+cmdline_compl     +lambda            +profile           +vim9script
+cmdline_hist      +langmap           -python            +viminfo
+cmdline_info      +libcall           +python3/dyn       +virtualedit
+comments          +linebreak         +quickfix          +visual
+conceal           +lispindent        +reltime           +visualextra
+cryptv            +listcmds          +rightleft         +vreplace
+cscope            +localmap          +ruby/dyn          +wildignore
+cursorbind        -lua               +scrollbind        +wildmenu
+cursorshape       +menu              +signs             +windows
+dialog_con        +mksession         +smartindent       +writebackup
+diff              +modify_fname      -sodium            -X11
+digraphs          +mouse             -sound             -xfontset
-dnd               -mouseshape        +spell             -xim
-ebcdic            +mouse_dec         +startuptime       -xpm
+emacs_tags        -mouse_gpm         +statusline        -xsmp
+eval              -mouse_jsbterm     -sun_workshop      -xterm_clipboard
+ex_extra          +mouse_netterm     +syntax            -xterm_save
+extra_search      +mouse_sgr         +tag_binary
-farsi             -mouse_sysmouse    -tag_old_static
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -g -O2 -D_REENTRANT -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: gcc -L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,-E -L/usr/local/lib -Wl,--as-needed -o vim -lm -ltinfo -lrt -ldl -Wl,-E -fstack-protector-strong -L/usr/local/lib -L/usr/lib/x86_64-linux-gnu/perl/5.30/CORE -lperl -ldl -lm -lpthread -lcrypt

Logs

gopls.3782930-5GiB-nonames.zip

[findleyr]

That gopls uses a lot of memory is a known (and difficult) problem. See for example #47855.

However, 5gb is a lot, and we should check whether that makes sense based on your usage. Can you tell me:

• Approximately how many .go files are in your workspace (this depends on which folder was resolved as your workspace folder).
• Do you experience this memory usage immediately, or does it take a while to build up? There was a large memory leak that will be fixed in the next gopls release (v0.9.2), so I'm curious if you may be experiencing that.

Unfortunately, at the moment 5gb is not unheard-of memory usage when working on large projects, such as Kubernetes. We're working on this, but it is a long-term project.

[codisms]

Unfortunately, I don't have a great way to answer the first question. The number is hard to know because of the libraries being referenced, but the project itself has 3500 Go files, though not all referenced in all parts of the code. One thing I can say is that we have a single file generated by gqlgen that has 256k lines in it and is probably a big culprit.

Initially loading the project causes the machine to spin for a good five minutes or so and it'll spike to 3-4 GB pretty easily. It'll settle around 2-3 after a while but spikes as the files are edited. Multiple files could be changed at the same time with multiple vim windows going on. I do have the gopls server running for quite a long time though.

[findleyr]

Thanks.

I expect you'll see it stay in the the ~3gb range after [email protected] is released. I'll take a look at the heap dump you've provided to see if I can understand this better.

[findleyr]

Putting this in the next milestone for additional triage.

[galileo-pkm]

Not much has changed with this release. gopls remains a memory hog. My "template" project without any real code and gopls eats up 2.2G. With a few medium size project in VSCode workspace it regularly goes over 8GB.

gopls v0.9.4, go.mod for the project and the diagnostics file attached if anyone cares. gopls.35536-2GiB-withnames.zip go.mod.gz

[switchupcb]

Do you experience this memory usage immediately, or does it take a while to build up? — @findleyr

I don't ever hit 1 GB of RAM (cause I'm not analyzing 256 KLOC). However, I noticed when running VSCode for a long time that gopls will freeze up and stop analyzing code. I kill the process (which shows higher memory consumption than usual) using Task Manager, which restarts gopls and it works for some time. Then, the cycle repeats.

Maybe a short term solution for the large projects is to have a mode that only analyzes specific files (i.e opened) or some way to exclude them (directory, whatever makes sense, etc).

Update: I hit 2GB on a 1500 line file.

[findleyr]

[email protected] significantly reduces gopls' memory usage, up to an order of magnitude on larger repositories.

Closing this as resolved. Please open a new issue if you are still encountering memory issues with the v0.12.0 release (the final release is scheduled for this week).