choco icon indicating copy to clipboard operation
choco copied to clipboard
verified publisher icon chocolatey

Chocolatey CLI occasionally appears to stop when running in a headless session on GCP

Open cfstras opened this issue 9 months ago • 7 comments

Checklist

  • [x] I confirm there are no unresolved issues reported on the Chocolatey Status page.
  • [x] I have verified this is the correct repository for opening this issue.
  • [x] I have verified no other issues exist related to my problem.
  • [x] I have verified this is not an issue for a specific package.
  • [x] I have verified this issue is not security related.
  • [x] I confirm I am using official, and not unofficial, or modified, Chocolatey products.

What You Are Seeing?

We have a GCP startup script running a chocolatey install. Sometimes, after a reboot, choco will print By installing, you accept licenses for the packages, and then hang indefinitely. Some infinite loop, because the memory gradually ramps up. I've last seen it using 19GiB memory and pinning one CPU core at ~90%.

This is on a box that already has these packages installed, and has run this exact choco command tens of times (on every reboot)

Image

What is Expected?

Chocolatey notices that the packages are already installed and exits.

How Did You Get This To Happen?

  1. Run choco install -y --no-progress 7zip vcredist-all ffmpeg gcloudsdk python313 directly after reboot
  2. In ~5% of cases, choco will hang and start consuming all memory.
  3. Killing choco or rebooting fixes the issue.

System Details

  • Operating System: Windows Server 2022 Datacenter 21H2 build 20348.3207
  • Windows PowerShell version: 5.1.20348.2849
  • Chocolatey CLI Version: 2.4.3
  • Chocolatey Licensed Extension version: Chocolatey v2.4.3 0 packages installed.
  • Chocolatey License type: Open Source
  • Terminal/Emulator: headless powershell (through GCEMetadataScripts) logging to event viewer

Installed Packages

7zip 24.9.0
7zip.install 24.9.0
chocolatey 2.4.3
chocolatey-compatibility.extension 1.0.0
chocolatey-core.extension 1.4.0
chocolatey-windowsupdate.extension 1.0.5
ffmpeg 7.1.0
gcloudsdk 512.0.0
KB2919355 1.0.20160915
KB2919442 1.0.20160915
KB2999226 1.0.20181019
KB3033929 1.0.5
KB3035131 1.0.3
python313 3.13.1
vcredist140 14.42.34438.20250221
vcredist2005 8.0.50727.619501
vcredist2008 9.0.30729.616104
vcredist2010 10.0.40219.32503
vcredist2012 11.0.61031.20230518
vcredist2013 12.0.40660.20180427
vcredist2015 14.0.24215.20170201
vcredist2017 14.16.27052
vcredist-all 1.0.1

Output Log

Chocolatey v2.4.3
Installing the following packages:
7zip;vcredist-all;ffmpeg;gcloudsdk;python313
By installing, you accept licenses for the packages.

---

From the log file -- the output for `--debug --verbose` also hangs after the last line: https://gist.github.com/cfstras/6a015ee723f8ff3afe4a33c413483e9f
I've included the last log from the previous invocation and the first line from after a reboot to show.

Additional Context

No response

cfstras avatar Mar 10 '25 10:03 cfstras

Have you tried running the installations of the packages 7zip vcredist-all ffmpeg gcloudsdk python313 separately to see which one may be causing the issue?

pauby avatar Mar 10 '25 10:03 pauby

I have yet to reproduce this in a lab setting - only happens in our automated production scripts so far.

Though I'm not sure whether this relates to a specific package, since it happens before any output that would hint that it's installing a package now?

cfstras avatar Mar 11 '25 12:03 cfstras

Though I'm not sure whether this relates to a specific package, since it happens before any output that would hint that it's installing a package now?

Without any reproducible steps, we can't rule anything out.

pauby avatar Mar 11 '25 13:03 pauby