cargo-quickinstall icon indicating copy to clipboard operation
cargo-quickinstall copied to clipboard

Please stop distributing malware :)

Open nabijaczleweli opened this issue 4 months ago • 22 comments

Downstream issue: https://github.com/nabijaczleweli/cargo-update/issues/305

Wherein a user downloads a release signed by you, and the binaries end up being malware. It would be convenient for me if you didn't do this since I don't particularly feel like dealing with more users getting malware in a way that appears like it came from me.

nabijaczleweli avatar Aug 26 '25 19:08 nabijaczleweli

I repro the submitter's experience with https://github.com/cargo-bins/cargo-quickinstall/releases/tag/cargo-update-18.0.0 -> cargo-update-18.0.0-x86_64-pc-windows-msvc.tar.gz -> cargo-install-update-config.exe -> https://www.virustotal.com/gui/file/aa69648ae6eb134aece49a7cf687a3aae3e8f9aae8f7baaf170491caf8e8fe14/detection

nabijaczleweli avatar Aug 26 '25 21:08 nabijaczleweli

Thank you for reporting this!

cc @alsuren we have to investigate this, I have no idea how that happens.

I have the feeling that something we installed on windows via scroop is compromised

NobodyXu avatar Aug 27 '25 09:08 NobodyXu

https://github.com/cargo-bins/cargo-quickinstall/actions/runs/17261615966

This is the CI for the windows build, I will take a look at the commands run

NobodyXu avatar Aug 27 '25 10:08 NobodyXu

Checked the CI, choco didn't install anything, which makes me think is one of our github account is compromised?

NobodyXu avatar Aug 27 '25 11:08 NobodyXu

Looking now.

alsuren avatar Aug 27 '25 12:08 alsuren

The artifact is available from https://github.com/cargo-bins/cargo-quickinstall/actions/runs/17261615966/ for post-mortem, and I have downloaded a copy for myself to look at later.

My leading theory is that the kaspersky warning is a false positive, but I will proceed as if it is not.

For now, I will

  • [x] disable all builds
  • [x] delete the tag associated with that build.
  • [x] look into the logs to see if there is anything suspicious
  • [x] find out if there is a way to run kapersky's trojan detector on our existing package repos to see if there is a pattern (help would be appreciated here).

alsuren avatar Aug 27 '25 12:08 alsuren

I have made a script to check (credit should go to Claude Sonnet 4 - https://github.com/cargo-bins/cargo-quickinstall/pull/443 )

It detects:

  • 17261615966 - cargo-install-update-config.exe https://www.virustotal.com/gui/file/e53e6def0a5f5989914fed682b631f1616c029b50bea21ff437e5d45a5aa4474
  • 17256188922 - els.exe https://www.virustotal.com/gui/file/fd2ece8e47216a4cc3d7599878b43d056ed20f0ae1133235e4d4148849086c67
  • 17255187714
    • rust-size.exe https://www.virustotal.com/gui/file/61d9870b7c7e3ea7452ac5e87a1ecab173b8aa19319f3440b3dcefec50192e6a
    • rust-profdata.exe https://www.virustotal.com/gui/file/3d6560eb9d9b3dc9338c46f2903c51ce56b80d1e22556c8ccb6e2f6dfe0e7d2a
    • rust-objdump.exe https://www.virustotal.com/gui/file/ec1e43b06316d51965f2fba0e266cc2fafac3dc7446e2441662f5eb9a85ce3b5
full json report: of 10 package builds, with 29 executables, 15 executables were flagged and 14 were clean
{
  "scan_timestamp": 1756308199.082388,
  "repo": "cargo-bins/cargo-quickinstall",
  "windows_builds_scanned": [
    "17257337803",
    "17256193567",
    "17256188922",
    "17255187714",
    "17255179000",
    "17255158233",
    "17255157323",
    "17252885295",
    "17251918191",
    "17251913163"
  ],
  "summary": {
    "builds_scanned": 10,
    "total_scanned": 29,
    "clean": 14,
    "flagged": 15
  },
  "results": [
    {
      "filename": "emmylua_ls.exe",
      "sha256": "b6a5e2ef00568a1d7dc05fa28013b4a8686df6258a70023b40b005489c6369d8",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 72,
      "total_engines": 72,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/b6a5e2ef00568a1d7dc05fa28013b4a8686df6258a70023b40b005489c6369d8"
    },
    {
      "filename": "els.exe",
      "sha256": "912f6aaebcd6b282d162c58ca663119b23b5327e7a56389041bb42a07f82527b",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 0,
      "total_engines": 0,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/912f6aaebcd6b282d162c58ca663119b23b5327e7a56389041bb42a07f82527b"
    },
    {
      "filename": "els.exe",
      "sha256": "fd2ece8e47216a4cc3d7599878b43d056ed20f0ae1133235e4d4148849086c67",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 69,
      "total_engines": 70,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/fd2ece8e47216a4cc3d7599878b43d056ed20f0ae1133235e4d4148849086c67"
    },
    {
      "filename": "cargo-readobj.exe",
      "sha256": "0a5c9d758f394da76e3b31928d379114354f9eb503a54b4e6475f64a9e90bd8a",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 72,
      "total_engines": 72,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/0a5c9d758f394da76e3b31928d379114354f9eb503a54b4e6475f64a9e90bd8a"
    },
    {
      "filename": "cargo-profdata.exe",
      "sha256": "c91142cc683513c434b429d8c1f91f9b359c75adf592160591fca1ef2ed0aa22",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 72,
      "total_engines": 72,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/c91142cc683513c434b429d8c1f91f9b359c75adf592160591fca1ef2ed0aa22"
    },
    {
      "filename": "cargo-strip.exe",
      "sha256": "bf7832386eda5ceef546c782b88db62594f4ee66841eabb5b639139286afd613",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 72,
      "total_engines": 72,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/bf7832386eda5ceef546c782b88db62594f4ee66841eabb5b639139286afd613"
    },
    {
      "filename": "rust-size.exe",
      "sha256": "61d9870b7c7e3ea7452ac5e87a1ecab173b8aa19319f3440b3dcefec50192e6a",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 71,
      "total_engines": 72,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/61d9870b7c7e3ea7452ac5e87a1ecab173b8aa19319f3440b3dcefec50192e6a"
    },
    {
      "filename": "cargo-nm.exe",
      "sha256": "c1af16f33653e1703e9b76b13d6368cd12c707b7839cc7fb7391ee51aeb85bf2",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 72,
      "total_engines": 72,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/c1af16f33653e1703e9b76b13d6368cd12c707b7839cc7fb7391ee51aeb85bf2"
    },
    {
      "filename": "rust-profdata.exe",
      "sha256": "3d6560eb9d9b3dc9338c46f2903c51ce56b80d1e22556c8ccb6e2f6dfe0e7d2a",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 71,
      "total_engines": 72,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/3d6560eb9d9b3dc9338c46f2903c51ce56b80d1e22556c8ccb6e2f6dfe0e7d2a"
    },
    {
      "filename": "rust-objdump.exe",
      "sha256": "ec1e43b06316d51965f2fba0e266cc2fafac3dc7446e2441662f5eb9a85ce3b5",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 71,
      "total_engines": 72,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/ec1e43b06316d51965f2fba0e266cc2fafac3dc7446e2441662f5eb9a85ce3b5"
    },
    {
      "filename": "cargo-objcopy.exe",
      "sha256": "05d23c33d0145e15308b54604d299b5417396a545e99b3725f0591a7309dd9de",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 71,
      "total_engines": 71,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/05d23c33d0145e15308b54604d299b5417396a545e99b3725f0591a7309dd9de"
    },
    {
      "filename": "rust-as.exe",
      "sha256": "79d28f4b364b8e6390aac5a4e1bc5112c387c11f3d97761baf3fb34f6226a5de",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 71,
      "total_engines": 72,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/79d28f4b364b8e6390aac5a4e1bc5112c387c11f3d97761baf3fb34f6226a5de"
    },
    {
      "filename": "rust-cov.exe",
      "sha256": "f84068f8cf8f0c557c2fa22f6340cf252b9a65383b6ed2e5bbea73d9510b623d",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 71,
      "total_engines": 72,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/f84068f8cf8f0c557c2fa22f6340cf252b9a65383b6ed2e5bbea73d9510b623d"
    },
    {
      "filename": "rust-ar.exe",
      "sha256": "2d8c0d16fd2b42e358fa2302b98bd1c049a665403a1813962fe7e79c422f8fc8",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 69,
      "total_engines": 70,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/2d8c0d16fd2b42e358fa2302b98bd1c049a665403a1813962fe7e79c422f8fc8"
    },
    {
      "filename": "cargo-objdump.exe",
      "sha256": "d24f139efce788ff9b7c19b9b9c423bbdd022fdf30f667cbf4617bf534cd901d",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 72,
      "total_engines": 72,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/d24f139efce788ff9b7c19b9b9c423bbdd022fdf30f667cbf4617bf534cd901d"
    },
    {
      "filename": "rust-strip.exe",
      "sha256": "5b2286ba6353882f5c4ee73da723477dd87483bca8a0c6642ec1dd97ba444cfd",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 71,
      "total_engines": 72,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/5b2286ba6353882f5c4ee73da723477dd87483bca8a0c6642ec1dd97ba444cfd"
    },
    {
      "filename": "rust-objcopy.exe",
      "sha256": "91623d317458150da81fb8ac7ff325d09989d8de53ee349161c75e246d7d4e49",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 71,
      "total_engines": 72,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/91623d317458150da81fb8ac7ff325d09989d8de53ee349161c75e246d7d4e49"
    },
    {
      "filename": "rust-nm.exe",
      "sha256": "6e66f3ed6ca2ef7680a761526371d40d159446f250f5f9ef371c853a02183c3e",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 71,
      "total_engines": 72,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/6e66f3ed6ca2ef7680a761526371d40d159446f250f5f9ef371c853a02183c3e"
    },
    {
      "filename": "rust-ld.exe",
      "sha256": "bd058f5040fcd75665ed3d63be5b3836d1a52ce001cbe2881e5e429658eceb4d",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 71,
      "total_engines": 72,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/bd058f5040fcd75665ed3d63be5b3836d1a52ce001cbe2881e5e429658eceb4d"
    },
    {
      "filename": "cargo-size.exe",
      "sha256": "b402c616e30bbb5ca6177470636f127de28c35ca98671c6730a6a85a9ce51db2",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 72,
      "total_engines": 72,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/b402c616e30bbb5ca6177470636f127de28c35ca98671c6730a6a85a9ce51db2"
    },
    {
      "filename": "cargo-cov.exe",
      "sha256": "2af1d97b2fa96f7d409a4f3e9718e8d89ed018395ee5cfff17a590f320c20538",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 72,
      "total_engines": 72,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/2af1d97b2fa96f7d409a4f3e9718e8d89ed018395ee5cfff17a590f320c20538"
    },
    {
      "filename": "rust-readobj.exe",
      "sha256": "e51d62c0e0108d32cc38c8d51777516681d2a7bac4530dd46552d428d7223f23",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 71,
      "total_engines": 72,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/e51d62c0e0108d32cc38c8d51777516681d2a7bac4530dd46552d428d7223f23"
    },
    {
      "filename": "rust-lld.exe",
      "sha256": "93128265ac9645642695fda2e510a0cc4ce1f8fe1ead72dcc650cb06d39c5c37",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 71,
      "total_engines": 72,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/93128265ac9645642695fda2e510a0cc4ce1f8fe1ead72dcc650cb06d39c5c37"
    },
    {
      "filename": "cargo-upgrades.exe",
      "sha256": "cd3de018c1fb0c56dae71d2440a5935584378746c64bd01eae98f7f27d2676a6",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 72,
      "total_engines": 72,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/cd3de018c1fb0c56dae71d2440a5935584378746c64bd01eae98f7f27d2676a6"
    },
    {
      "filename": "build_re_types.exe",
      "sha256": "d6610244273450666e7684089de7d4e55d9663a0333376419e973bdefa5e927b",
      "malicious": 1,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 67,
      "total_engines": 68,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/d6610244273450666e7684089de7d4e55d9663a0333376419e973bdefa5e927b"
    },
    {
      "filename": "build_re_types.exe",
      "sha256": "abc7bb22df2680812210bc85aae35536ebb6987acda3a7cb704400509bd0b705",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 72,
      "total_engines": 72,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/abc7bb22df2680812210bc85aae35536ebb6987acda3a7cb704400509bd0b705"
    },
    {
      "filename": "doxx.exe",
      "sha256": "5d49d46bf19ebd2b73ee8c73cabd2104d71c5ccd6a3d1d36354743aa6dcea0a6",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 72,
      "total_engines": 72,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/5d49d46bf19ebd2b73ee8c73cabd2104d71c5ccd6a3d1d36354743aa6dcea0a6"
    },
    {
      "filename": "test.exe",
      "sha256": "ac59d93ff123d9d8a449c376c5f90f6eabfb1a4eda11689d3c24ce203965814f",
      "malicious": 0,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 72,
      "total_engines": 72,
      "is_safe": true,
      "scan_url": "https://www.virustotal.com/gui/file/ac59d93ff123d9d8a449c376c5f90f6eabfb1a4eda11689d3c24ce203965814f"
    },
    {
      "filename": "cranelift-assembler-x64.exe",
      "sha256": "731065cdf6b599628da11871b90951cd8bbffbb4155e5317b19088b674acd06c",
      "malicious": 2,
      "suspicious": 0,
      "harmless": 0,
      "undetected": 70,
      "total_engines": 72,
      "is_safe": false,
      "scan_url": "https://www.virustotal.com/gui/file/731065cdf6b599628da11871b90951cd8bbffbb4155e5317b19088b674acd06c"
    }
  ]
}

alsuren avatar Aug 27 '25 14:08 alsuren

Out of an abundance of caution, I am going to

  • [x] delete all tags and all associated release artefacts.

If we can prove that these are all false positives then we can potentially recreate the tags from the build artefacts. Otherwise, I think we will need to take a step back, make sure our builds are correctly isolated, then start again and build everything up from scratch.

  • [ ] We should probably also try to inform everyone who has used our releases in their CI. I'm not sure how to do that though.

alsuren avatar Aug 27 '25 18:08 alsuren

Well, at least one of my CI setups started to fail, in which case informing will be easier...

It seems relatively likely that this Acronis (Static ML) thing is way too trigger happy.

djc avatar Aug 27 '25 20:08 djc

Well, at least one of my CI setups started to fail, in which case informing will be easier...

Hrm. I didn't think that there were cases where cargo-binstall could succeed where cargo install would fail (in this case because your rustc is too old). This might cause more havoc than I expected.

It seems relatively likely that this Acronis (Static ML) thing is way too trigger happy.

This is what I'm hoping too. How should I go about proving it?

alsuren avatar Aug 27 '25 20:08 alsuren

It seems relatively likely that this Acronis (Static ML) thing is way too trigger happy.

This is what I'm hoping too. How should I go about proving it?

I'm not sure. Ultimately this needs something like reproducible builds, I guess, so you can prove that binaries hosted here are the same built somewhere in a more isolated environment.

djc avatar Aug 27 '25 20:08 djc

cargo-bins/cargo-quickinstall/actions/runs/17261615966

This is the CI for the windows build, I will take a look at the commands run

That appears to be aarch64, the x86-64 build was https://github.com/cargo-bins/cargo-quickinstall/actions/runs/17242878973 which I've confirmed the cargo-install-update-config.exe file in its artifacts has the same hash as the virus total link.

EDIT: Which I believe rules out account compromise, the file on the release was generated on the runner (unless there's some way I don't know about to replace artifacts post-run). I think leaving the potential causes at:

  • the runner itself was compromised before the job started
  • the build steps somehow compromised it
  • they're all false positive detections

Nemo157 avatar Aug 28 '25 07:08 Nemo157

Ideally, I would like to rule out "the runner itself was compromised before the job started" and "the build steps somehow compromised it" a bit harder before I conclude that they're all false positives.

Routes that I would like to double check include cache poisoning (I was very careful to avoid using any caches in the original design, so it may be easy to rule this out if nothing has changed, but I haven't touched this codebase in quite a long time), token privileges (I think I locked this down before) and deploy keys/secrets that might enable information sharing.

Once I'm happy that this is a false alarm, it looks like we have a bunch of draft releases with the artefacts still intact. I will investigate whether these can be promoted to real releases programmatically.

Realistically I'm unlikely to have the brain space to go through all of this to my satisfaction until Sunday afternoon (next week is also going to be busy).

If someone can come up with a set of steps to reproduce the flagged binaries on a clean windows box (I don't have access to a windows box, but I can use https://github.com/mxschmitt/action-tmate from a clean github account) then that might help.

alsuren avatar Aug 28 '25 09:08 alsuren

Routes that I would like to double check include cache poisoning (I was very careful to avoid using any caches in the original design, so it may be easy to rule this out if nothing has changed, but I haven't touched this codebase in quite a long time)

Can confirm that I didn't add any caching for build-package.yml, so it cannot be cache poisoning.

And the upload is done in a separate job, the build job doesn't have any privileges to upload anything to begin with.

NobodyXu avatar Aug 28 '25 10:08 NobodyXu

I tried setting up a similar GHA build at https://github.com/Nemo157/quickinstall-cargo-update-test-build/actions/runs/17318317272.

The result got 3 heuristic hits, but not the maybe non-heuristic hits https://www.virustotal.com/gui/file/c544b2aefe5c733ca100f90275fddb5319b31d5164ecfe0436ed5a62b6d8148a/details.

Interestingly it is the exact same size with the same section layout, so maybe if someone knows how to do a useful binary diff of windows executables that might show something.

Nemo157 avatar Aug 29 '25 08:08 Nemo157

so maybe if someone knows how to do a useful binary diff of windows executables that might show something.

https://diffoscope.org/ may work. There is an online version: https://try.diffoscope.org/ (I believe running diffoscope locally on untrusted executables is a bad idea) You may also want to try building on a repo whose length combined with your username length matches this repo as differing build path lengths are likely to result in cascading differences.

bjorn3 avatar Aug 29 '25 13:08 bjorn3

https://try.diffoscope.org/utdabpfxceuv.html

(-1 was the artifact from here, -2 from my job). Seems like a very small diff.

Nemo157 avatar Aug 29 '25 13:08 Nemo157

Yeah..just looks like LLVM generates stuff differently, doesn't look like some malicious code is being slipped in

NobodyXu avatar Aug 29 '25 13:08 NobodyXu

Thanks to everyone for your help in establishing that this was a false alarm.

It seems like there is a programmatic way to undraft releases. I'm kicking off the process now.

I probably won't re-enable new package builds until next week.

@nabijaczleweli this means that the package that your users were complaining about is going to come back. Would you be able to follow https://github.com/cargo-bins/cargo-binstall/blob/main/SUPPORT.md to point binstall at a set of binaries that you've built yourself as a work-around? I haven't looked into how your tool works, so sorry if you've already done this, or it's not an appropriate solution.

alsuren avatar Aug 30 '25 16:08 alsuren

Still not interested, but the quickinstall distribution seems to work for most users most of the time, so I leave that in your capable and sovereign hands. Thanks for the eval.

nabijaczleweli avatar Aug 30 '25 18:08 nabijaczleweli

I'm going to keep this issue open until I've dealt with the fallout.

Status update for anyone following along at home: I re-enabled the builders and undeleted the tags (I also pushed the script for that to #443). Unfortunately, that leaves us with a bunch of releases that think they have assets attached to them but actually don't, e.g. https://github.com/cargo-bins/cargo-quickinstall/releases/tag/ripgrep-14.0.0 .

  • [ ] see if there is a way to quickly restore the assets from recent builds and recalculate the signatures for them (this would be better and also less computationally expensive, but I have never touched this code before so maybe I just won't do this)
  • [x] kick off the builders again and get them to rebuild everything from scratch (this will cause the hashes of the released binaries to change, which is something that I was hoping to avoid, but I think we're just going to have to live with it)

alsuren avatar Aug 30 '25 18:08 alsuren

see if there is a way to quickly restore the assets from recent builds and recalculate the signatures for them

I think the build artifacts also include the signature?

NobodyXu avatar Aug 31 '25 03:08 NobodyXu