jfrog-cli
jfrog-cli copied to clipboard
jfrog
Possible memory leak or extensive usage of in memory manipulations when running `jf yarn install` on larger projects with node-gyp
Describe the bug
We are currently experiencing out-of-memory errors when using the jf cli for installing dependencies on larger projects with the yarn subcommand jf yarn install. It seems to be a problem with the functionality related to the collection of build info.
When running directly the yarn install command without passing through jf cli the installation process runs fine.
We are running these actions through a CI/CD pipeline on a RHEL8 host with 8 GB of RAM.
Current behavior
... JFrog CLI version: 2.56.0 ##[debug]set JFROG_CLI_TASK_SELECTED_VERSION_AZURE=2.56.0 ... 16:43:36 [Debug] JFrog CLI version: 2.56.0 16:43:36 [Debug] OS/Arch: linux/amd64 ... ##[debug]JFROG_CLI_TASK_SELECTED_VERSION_AZURE=2.56.0 Executing JFrog CLI Command: /__t/jf/2.56.0/x64/jf yarn-config --repo-resolve=npm-prd 16:43:37 [Debug] JFrog CLI version: 2.56.0 16:43:37 [Debug] OS/Arch: linux/amd64 16:43:37 [Info] yarn build config successfully created. ##[debug]JFROG_CLI_TASK_SELECTED_VERSION_AZURE=2.56.0 Executing JFrog CLI Command: /__t/jf/2.56.0/x64/jf yarn install --build-name='aProject-aRepo' --build-number='20240425.4-r1' 16:43:37 [Debug] JFrog CLI version: 2.56.0 16:43:37 [Debug] OS/Arch: linux/amd64 16:43:37 [Info] Running Yarn... 16:43:37 [Debug] Preparing to read the config file /__w/104/s/.jfrog/projects/yarn.yaml 16:43:37 [Debug] Found resolver in the config file /__w/104/s/.jfrog/projects/yarn.yaml 16:43:37 [Debug] Preparing prerequisites. 16:43:37 [Debug] Found Yarn executable at: /opt/node-v20.12.2-linux-x64/bin/yarn 16:43:37 [Debug] Working directory set to: /__w/104/s 16:43:37 [Debug] Saving build general details at: /tmp/jfrog/builds/f5f6ce19cbe75fe94655d341be05664b7ecfe61e55f546aedeef6639dd3161e9/partials 16:43:37 [Debug] Found Yarn executable at: /opt/node-v20.12.2-linux-x64/bin/yarn 16:43:38 [Debug] Sending HTTP GET request to: https://myserver/artifactory/api/system/version 16:43:38 [Debug] Artifactory response: 200 16:43:38 [Debug] JFrog Artifactory version is: 7.77.9 16:43:38 [Debug] Sending npm auth request 16:43:38 [Debug] Sending HTTP GET request to: https://myserver/artifactory/api/npm/auth 16:43:38 [Debug] Sending HTTP GET request to: https://myserver/artifactory/api/repositories/npm-prd 16:43:38 [Info] Preparing for dependencies information collection... For the first run of the build, the dependencies collection may take a few minutes. Subsequent runs should be faster. 16:43:38 [Debug] Searching Artifactory using AQL query: items.find({"repo": "artifactory-build-info","path": {"$match": "aProject-aRepo"}}).sort({"$desc":["created"]}).limit(1) 16:43:38 [Debug] Sending HTTP POST request to: https://myserver/artifactory/api/search/aql 16:43:38 [Debug] Artifactory response: 200 16:43:38 [Debug] Streaming data to file... 16:43:38 [Debug] Finished streaming data successfully. 16:43:38 [Debug] A build-name: <aProject-aRepo> with a build-number: <LATEST> could not be found in Artifactory. 16:43:38 [Debug] The file /__w/104/s/.yarnrc.yml was backed up successfully to jfrog.yarnrc.backup ➤ YN0000: ┌ Resolution step ➤ YN0002: │ @aws-sdk/token-providers@npm:3.556.0 doesn't provide @aws-sdk/credential-provider-node (pd256c), requested by @aws-sdk/client-sso-oidc ... ➤ YN0000: └ Completed in 1m 14s ➤ YN0000: Done with warnings in 2m 57s 16:46:41 [Debug] Fetching checksums for validate.io-array:1.0.6 16:46:41 [Debug] Searching Artifactory using AQL query: items.find({"@npm.name":"validate.io-array","$or": [{"@npm.version":"1.0.6"},{"@npm.version":"v1.0.6"}]}).include("name","repo","path","actual_sha1","actual_md5","sha256") ... 16:46:47 [Debug] @babel/plugin-bugfix-firefox-class-in-computed-class-key:7.24.4 was found in Artifactory. Name: plugin-bugfix-firefox-class-in-computed-class-key-7.24.4.tgz SHA-1: 6125f0158543fb4edf1c22f322f3db67f21cb3e1 MD5: 260e23bd02686c766c2dd86d66ebc7ee fatal error: runtime: out of memory
runtime stack: runtime.throw({0x115fe9b?, 0x0?}) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/panic.go:1047 +0x5d fp=0xc000071df8 sp=0xc000071dc8 pc=0x436edd runtime.sysMapOS(0xc25c000000, 0x100000000?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mem_linux.go:187 +0x11b fp=0xc000071e40 sp=0xc000071df8 pc=0x41809b runtime.sysMap(0x1c71fa0?, 0xc3ffffffff?, 0x1c82138?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mem.go:142 +0x35 fp=0xc000071e70 sp=0xc000071e40 pc=0x417a75 runtime.(*mheap).grow(0x1c71fa0, 0x80000?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mheap.go:1522 +0x252 fp=0xc000071ee8 sp=0xc000071e70 pc=0x428232 runtime.(*mheap).allocSpan(0x1c71fa0, 0x80000, 0x0, 0x1?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mheap.go:1243 +0x1b7 fp=0xc000071f80 sp=0xc000071ee8 pc=0x427977 runtime.(*mheap).alloc.func1() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mheap.go:961 +0x65 fp=0xc000071fc8 sp=0xc000071f80 pc=0x427425 runtime.systemstack() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:496 +0x49 fp=0xc000071fd0 sp=0xc000071fc8 pc=0x467e69
goroutine 1 [running]: runtime.systemstack_switch() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:463 fp=0xc0000b4f88 sp=0xc0000b4f80 pc=0x467e00 runtime.(*mheap).alloc(0x1c71fa0?, 0x80000?, 0x1?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mheap.go:955 +0x65 fp=0xc0000b4fd0 sp=0xc0000b4f88 pc=0x427365 runtime.(*mcache).allocLarge(0xc0000061a0?, 0x100000000, 0x1) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mcache.go:234 +0x85 fp=0xc0000b5018 sp=0xc0000b4fd0 pc=0x416b05 runtime.mallocgc(0x100000000, 0x0, 0x0) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/malloc.go:1053 +0x4fe fp=0xc0000b5080 sp=0xc0000b5018 pc=0x40d9fe runtime.growslice(0x0, 0x5b203a2273656c75?, 0x6d73402220202020?, 0x696d732f79687469?, 0x65696c632d796874?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/slice.go:266 +0x4a8 fp=0xc0000b50e0 sp=0xc0000b5080 pc=0x44eca8 bytes.growSlice({0xc1dbd4c000, 0x80000000, 0xc0000b51a8?}, 0x527536?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/bytes/buffer.go:240 +0x9b fp=0xc0000b5160 sp=0xc0000b50e0 pc=0x50169b bytes.(*Buffer).grow(0xc0000b52f0, 0x1) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/bytes/buffer.go:142 +0x14f fp=0xc0000b5198 sp=0xc0000b5160 pc=0x50108f bytes.(*Buffer).WriteByte(0xc0000b52f0, 0x74) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/bytes/buffer.go:280 +0x52 fp=0xc0000b51b8 sp=0xc0000b5198 pc=0x501932 encoding/json.Indent(0xc0000b52f0, {0xc13f9e8000, 0x5c362d2a, 0x485bcb?}, {0x0, 0x0}, {0x111f48c, 0x2}) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/encoding/json/indent.go:105 +0x33a fp=0xc0000b5250 sp=0xc0000b51b8 pc=0x5278da github.com/jfrog/build-info-go/build.(*Build).SaveBuildInfo(0xc000424000, 0xc0003fa210?) /root/go/pkg/mod/github.com/jfrog/[email protected]/build/build.go:216 +0xac fp=0xc0000b5330 sp=0xc0000b5250 pc=0xcd276c github.com/jfrog/build-info-go/build.(*YarnModule).Build(0xc000426070) /root/go/pkg/mod/github.com/jfrog/[email protected]/build/yarn.go:81 +0x1d5 fp=0xc0000b5430 sp=0xc0000b5330 pc=0xcde855 github.com/jfrog/jfrog-cli-core/v2/artifactory/commands/yarn.(*YarnCommand).Run(0xc000344a00) /root/go/pkg/mod/github.com/jfrog/jfrog-cli-core/[email protected]/artifactory/commands/yarn/yarn.go:111 +0x391 fp=0xc0000b55b8 sp=0xc0000b5430 pc=0xdcacd1 github.com/jfrog/jfrog-cli-core/v2/common/commands.Exec({0x1372210?, 0xc000344a00}) /root/go/pkg/mod/github.com/jfrog/jfrog-cli-core/[email protected]/common/commands/command.go:26 +0xa8 fp=0xc0000b55f0 sp=0xc0000b55b8 pc=0xc99368 github.com/jfrog/jfrog-cli/buildtools.YarnCmd(0xc00016edc0) /var/jenkins_home/workspace/eco-system/release/jfrog-cli-release/temp/jfrog-cli/buildtools/cli.go:524 +0xfd fp=0xc0000b5638 sp=0xc0000b55f0 pc=0xe731bd github.com/urfave/cli.HandleAction({0xf9e320?, 0x1202b58?}, 0x4?) /root/go/pkg/mod/github.com/urfave/[email protected]/app.go:524 +0x50 fp=0xc0000b5650 sp=0xc0000b5638 pc=0xc85010 github.com/urfave/cli.Command.Run({{0x11215d5, 0x4}, {0x0, 0x0}, {0x0, 0x0, 0x0}, {0x1150459, 0x12}, {0x0, ...}, ...}, ...) /root/go/pkg/mod/github.com/urfave/[email protected]/command.go:175 +0x67b fp=0xc0000b5880 sp=0xc0000b5650 pc=0xc860fb github.com/urfave/cli.(*App).Run(0xc00014a700, {0xc0000340a0, 0x5, 0x5}) /root/go/pkg/mod/github.com/urfave/[email protected]/app.go:277 +0xb87 fp=0xc0000b5e60 sp=0xc0000b5880 pc=0xc82b07 main.execMain() /var/jenkins_home/workspace/eco-system/release/jfrog-cli-release/temp/jfrog-cli/main.go:141 +0x4b3 fp=0xc0000b5f30 sp=0xc0000b5e60 pc=0xedb413 main.main() /var/jenkins_home/workspace/eco-system/release/jfrog-cli-release/temp/jfrog-cli/main.go:80 +0x25 fp=0xc0000b5f80 sp=0xc0000b5f30 pc=0xedae85 runtime.main() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:250 +0x207 fp=0xc0000b5fe0 sp=0xc0000b5f80 pc=0x439807 runtime.goexit() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000b5fe8 sp=0xc0000b5fe0 pc=0x469ee1
goroutine 2 [force gc (idle), 3 minutes]: runtime.gopark(0x13469b6472427a?, 0x0?, 0x0?, 0x0?, 0x0?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:381 +0xd6 fp=0xc000062fb0 sp=0xc000062f90 pc=0x439c36 runtime.goparkunlock(...) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:387 runtime.forcegchelper() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:305 +0xb0 fp=0xc000062fe0 sp=0xc000062fb0 pc=0x439a70 runtime.goexit() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000062fe8 sp=0xc000062fe0 pc=0x469ee1 created by runtime.init.6 /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:293 +0x25
goroutine 3 [GC sweep wait]: runtime.gopark(0x1c57a01?, 0x0?, 0x0?, 0x0?, 0x0?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:381 +0xd6 fp=0xc000063780 sp=0xc000063760 pc=0x439c36 runtime.goparkunlock(...) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:387 runtime.bgsweep(0x0?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgcsweep.go:319 +0xde fp=0xc0000637c8 sp=0xc000063780 pc=0x42427e runtime.gcenable.func1() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgc.go:178 +0x26 fp=0xc0000637e0 sp=0xc0000637c8 pc=0x4194c6 runtime.goexit() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000637e8 sp=0xc0000637e0 pc=0x469ee1 created by runtime.gcenable /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgc.go:178 +0x6b
goroutine 4 [GC scavenge wait]: runtime.gopark(0x13467f31f3dd9d?, 0x3ba7e78a?, 0x0?, 0x0?, 0x0?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:381 +0xd6 fp=0xc000063f70 sp=0xc000063f50 pc=0x439c36 runtime.goparkunlock(...) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:387 runtime.(*scavengerState).park(0x1c58460) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc000063fa0 sp=0xc000063f70 pc=0x422133 runtime.bgscavenge(0x0?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgcscavenge.go:633 +0x65 fp=0xc000063fc8 sp=0xc000063fa0 pc=0x422725 runtime.gcenable.func2() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgc.go:179 +0x26 fp=0xc000063fe0 sp=0xc000063fc8 pc=0x419466 runtime.goexit() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000063fe8 sp=0xc000063fe0 pc=0x469ee1 created by runtime.gcenable /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgc.go:179 +0xaa
goroutine 5 [finalizer wait]: runtime.gopark(0x0?, 0xc00050eb88?, 0xe0?, 0x0?, 0x1000000010?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:381 +0xd6 fp=0xc000102e28 sp=0xc000102e08 pc=0x439c36 runtime.runfinq() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mfinal.go:193 +0x107 fp=0xc000102fe0 sp=0xc000102e28 pc=0x418507 runtime.goexit() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000102fe8 sp=0xc000102fe0 pc=0x469ee1 created by runtime.createfing /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mfinal.go:163 +0x45
goroutine 6 [chan send, 3 minutes]: runtime.gopark(0x115eac1?, 0x16?, 0x78?, 0x46?, 0xa77a71?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:381 +0xd6 fp=0xc0000645e8 sp=0xc0000645c8 pc=0x439c36 runtime.chansend(0xc00037ff20, 0x135fc08, 0x1, 0x115eac1?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/chan.go:259 +0x42e fp=0xc000064670 sp=0xc0000645e8 pc=0x40618e runtime.chansend1(0x0?, 0x0?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/chan.go:145 +0x1d fp=0xc0000646a0 sp=0xc000064670 pc=0x405d3d github.com/jfrog/jfrog-cli-core/v2/common/commands.signalReportUsageFinished(0x0?) /root/go/pkg/mod/github.com/jfrog/jfrog-cli-core/[email protected]/common/commands/command.go:61 +0x25 fp=0xc0000646c0 sp=0xc0000646a0 pc=0xc99a25 github.com/jfrog/jfrog-cli-core/v2/common/commands.reportUsage.func1() /root/go/pkg/mod/github.com/jfrog/jfrog-cli-core/[email protected]/common/commands/command.go:33 +0x26 fp=0xc0000646d8 sp=0xc0000646c0 pc=0xc999c6 github.com/jfrog/jfrog-cli-core/v2/common/commands.reportUsage({0x1372210, 0xc000344a00}, 0x0?) /root/go/pkg/mod/github.com/jfrog/jfrog-cli-core/[email protected]/common/commands/command.go:57 +0x524 fp=0xc0000647b8 sp=0xc0000646d8 pc=0xc99944 github.com/jfrog/jfrog-cli-core/v2/common/commands.Exec.func1() /root/go/pkg/mod/github.com/jfrog/jfrog-cli-core/[email protected]/common/commands/command.go:24 +0x2e fp=0xc0000647e0 sp=0xc0000647b8 pc=0xc993ee runtime.goexit() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000647e8 sp=0xc0000647e0 pc=0x469ee1 created by github.com/jfrog/jfrog-cli-core/v2/common/commands.Exec /root/go/pkg/mod/github.com/jfrog/jfrog-cli-core/[email protected]/common/commands/command.go:24 +0x98
goroutine 33 [GC worker (idle)]: runtime.gopark(0x1346b40c2777bf?, 0x3?, 0xa5?, 0x4e?, 0x0?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:381 +0xd6 fp=0xc00005e750 sp=0xc00005e730 pc=0x439c36 runtime.gcBgMarkWorker() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgc.go:1275 +0xf1 fp=0xc00005e7e0 sp=0xc00005e750 pc=0x41b231 runtime.goexit() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00005e7e8 sp=0xc00005e7e0 pc=0x469ee1 created by runtime.gcBgMarkStartWorkers /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgc.go:1199 +0x25
goroutine 49 [GC worker (idle)]: runtime.gopark(0x1c8a720?, 0x1?, 0x36?, 0x19?, 0x0?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:381 +0xd6 fp=0xc000508750 sp=0xc000508730 pc=0x439c36 runtime.gcBgMarkWorker() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgc.go:1275 +0xf1 fp=0xc0005087e0 sp=0xc000508750 pc=0x41b231 runtime.goexit() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0005087e8 sp=0xc0005087e0 pc=0x469ee1 created by runtime.gcBgMarkStartWorkers /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgc.go:1199 +0x25
goroutine 50 [GC worker (idle)]: runtime.gopark(0x1346b40bf62ae0?, 0x3?, 0xee?, 0x71?, 0x0?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:381 +0xd6 fp=0xc000508f50 sp=0xc000508f30 pc=0x439c36 runtime.gcBgMarkWorker() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgc.go:1275 +0xf1 fp=0xc000508fe0 sp=0xc000508f50 pc=0x41b231 runtime.goexit() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000508fe8 sp=0xc000508fe0 pc=0x469ee1 created by runtime.gcBgMarkStartWorkers /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgc.go:1199 +0x25
goroutine 51 [GC worker (idle)]: runtime.gopark(0x1346abf0b553d2?, 0x1?, 0xc8?, 0x57?, 0x0?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:381 +0xd6 fp=0xc000509750 sp=0xc000509730 pc=0x439c36 runtime.gcBgMarkWorker() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgc.go:1275 +0xf1 fp=0xc0005097e0 sp=0xc000509750 pc=0x41b231 runtime.goexit() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0005097e8 sp=0xc0005097e0 pc=0x469ee1 created by runtime.gcBgMarkStartWorkers /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/mgc.go:1199 +0x25
goroutine 42 [chan receive]: runtime.gopark(0xc00053a420?, 0xf8a360?, 0x0?, 0x82?, 0x7fece7204118?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/proc.go:381 +0xd6 fp=0xc000106ea8 sp=0xc000106e88 pc=0x439c36 runtime.chanrecv(0xc000046a80, 0xc000106fc0, 0x1) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/chan.go:583 +0x49d fp=0xc000106f38 sp=0xc000106ea8 pc=0x406f9d runtime.chanrecv2(0xc01f03e200?, 0xc000106fd0?) /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/chan.go:447 +0x18 fp=0xc000106f60 sp=0xc000106f38 pc=0x406ad8 github.com/jfrog/jfrog-cli-core/v2/artifactory/commands/yarn.(*YarnCommand).Run.func1() /root/go/pkg/mod/github.com/jfrog/jfrog-cli-core/[email protected]/artifactory/commands/yarn/yarn.go:95 +0x45 fp=0xc000106fe0 sp=0xc000106f60 pc=0xdcb425 runtime.goexit() /var/jenkins_home/tools/org.jenkinsci.plugins.golang.GolangInstallation/go-1.20.14/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000106fe8 sp=0xc000106fe0 pc=0x469ee1 created by github.com/jfrog/jfrog-cli-core/v2/artifactory/commands/yarn.(*YarnCommand).Run /root/go/pkg/mod/github.com/jfrog/jfrog-cli-core/[email protected]/artifactory/commands/yarn/yarn.go:94 +0x205 ##[debug]task result: Failed ##[error]Error: Command failed: /__t/jf/2.56.0/x64/jf yarn install --build-name='aProject-aRepo' --build-number='20240425.4-r1' ##[debug]Processed: ##vso[task.issue type=error;]Error: Command failed: /__t/jf/2.56.0/x64/jf yarn install --build-name='aProject-aRepo' --build-number='20240425.4-r1' ##[debug]Processed: ##vso[task.complete result=Failed;]Error: Command failed: /__t/jf/2.56.0/x64/jf yarn install --build-name='aProject-aRepo' --build-number='20240425.4-r1' Executing JFrog CLI Command: /__t/jf/2.56.0/x64/jf c remove "aRepo_20240425.4_jfrog_cli_cmd_1714063416845" --quiet 16:47:32 [Debug] JFrog CLI version: 2.56.0 16:47:32 [Debug] OS/Arch: linux/amd64 16:47:32 [Debug] Locking config file to run config Delete command. ...
Reproduction steps
Have a large nodejs project with several workspaces, collect build info and run jf yarn install
Expected behavior
Not require more than 8Gb of RAM to install package dependencies using yarn
JFrog CLI version
2.56.0
Operating system type and version
Linux RHEL8
JFrog Artifactory version
7.77.9
JFrog Xray version
3.94.4
Hi @carneirofc ,
We weren't able to reproduce this issue so far, but we will keep looking into it to try and improve the memory consumption.
It would be helpful if you could provide the yarn version you are using, and possibly an example project to reproduce with.
Please note that the jf yarn install command supports a --threads option which you can use to limit the number of working threads for build-info collection (default is 3).
Thanks
