rustup icon indicating copy to clipboard operation
rustup copied to clipboard
verified publisher icon rust-lang

`rustup update` panic while cleaning cache

Open andrewsonin opened this issue 4 years ago • 25 comments

Problem

info: syncing channel updates for 'stable-x86_64-apple-darwin'
info: checking for self-updates

  stable-x86_64-apple-darwin unchanged - rustc 1.56.1 (59eed8a2a 2021-11-01)

info: cleaning up downloads & tmp directories
thread 'main' panicked at 'Unable to clean up /Users/andrewsonin/.rustup/tmp: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }', src/utils/utils.rs:642:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Steps

  1. curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
  2. rustup target add x86_64-unknown-linux-musl x86_64-pc-windows-gnu
  3. rustup update

Possible Solution(s)

No response

Notes

No response

Rustup version

rustup 1.24.3 (ce5817a94 2021-05-31)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active `rustc` version is `rustc 1.56.1 (59eed8a2a 2021-11-01)`

Installed toolchains

Default host: x86_64-apple-darwin
rustup home:  /Users/andrewsonin/.rustup

installed targets for active toolchain
--------------------------------------

x86_64-apple-darwin
x86_64-pc-windows-gnu
x86_64-unknown-linux-musl

active toolchain
----------------

stable-x86_64-apple-darwin (default)
rustc 1.56.1 (59eed8a2a 2021-11-01)

Operating system

macOS Monterey 12.0.1

andrewsonin avatar Nov 09 '21 09:11 andrewsonin

Just found that I can't reproduce the bug following the steps outlined here. Hm...

andrewsonin avatar Nov 09 '21 10:11 andrewsonin

We typically see permission errors in IO only on Windows - I have two guesses as to what this could be, neither are easy to reproduce reliably:

  1. You may be running some kind of virus scanner which decided to lock and scan the temp directory right before we tried to remove it
  2. You may be running multiple rustup operations simultaneously (e.g. via an IDE as well as the CLI) and these can conflict if you're not careful.

Do either of those ring bells?

kinnison avatar Nov 11 '21 10:11 kinnison

@kinnison No, none of those ring bells.

andrewsonin avatar Nov 15 '21 08:11 andrewsonin

What's the ownership on the two last directories in the path that fails to delete?

On Mon, 15 Nov 2021, 09:08 Andrew Sonin, @.***> wrote:

@kinnison https://github.com/kinnison No, none of those ring bells.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/rust-lang/rustup/issues/2895#issuecomment-968631037, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADZ7XVXWL4WJJUUM3YQLVTUMC5Y7ANCNFSM5HU3FWVQ .

rbtcollins avatar Nov 15 '21 08:11 rbtcollins

@rbtcollins

Since then I have completely reinstalled rustup, so there is no way for me to recover this information.

However, the command ls -shal /Users/andrewsonin/.rustup/tmp now outputs the following:

total 0
0 drwxr-xr-x 2 andrewsonin staff 64B Nov 15 13:23 .
0 drwxr-xr-x 7 andrewsonin staff 224B Nov 9 13:13 ..

I think the permissions could not be different from the current ones, since I worked with rustup exclusively from the terminal without using sudo.

andrewsonin avatar Nov 15 '21 11:11 andrewsonin

Thanks, though we would have needed the original numbers not the re-installed ones to be able to exclude permissions as a cause.

On a non-windows OS, without permissions preventing deletion the only remaining cause is a disconnected network drive, or a kernel security module of some sort intercepting the operation.

-Rob

On Mon, 15 Nov 2021 at 12:28, Andrew Sonin @.***> wrote:

@rbtcollins https://github.com/rbtcollins

Since then I have completely reinstalled rustup, so there is no way for me to recover this information.

However, the command ls -shal /Users/andrewsonin/.rustup/tmp now outputs the following:

total 0 0 drwxr-xr-x 2 andrewsonin staff 64B Nov 15 13:23 . 0 drwxr-xr-x 7 andrewsonin staff 224B Nov 9 13:13 ..

I think the permissions could not be different from the current ones, since I worked with rustup exclusively from the terminal without using sudo.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rust-lang/rustup/issues/2895#issuecomment-968815998, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADZ7XWBQHEV6YQBPXY2G6LUMDVHDANCNFSM5HU3FWVQ .

rbtcollins avatar Nov 15 '21 17:11 rbtcollins

I can repeatedly reproduce this right now on Mac OS 12.1 ARM64, will investigate.

rustup update output:

$ rustup update
info: syncing channel updates for 'stable-aarch64-apple-darwin'
info: syncing channel updates for 'stable-x86_64-apple-darwin'
info: syncing channel updates for 'nightly-aarch64-apple-darwin'
info: checking for self-updates

   stable-aarch64-apple-darwin unchanged - rustc 1.57.0 (f1edd0429 2021-11-29)
    stable-x86_64-apple-darwin unchanged - rustc 1.57.0 (f1edd0429 2021-11-29)
  nightly-aarch64-apple-darwin unchanged - rustc 1.59.0-nightly (c5ecc1570 2021-12-15)

info: cleaning up downloads & tmp directories
thread 'main' panicked at 'Unable to clean up /Users/hans/.rustup/tmp: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }', src/utils/utils.rs:642:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The directory looks like this:

$ ls -alR /Users/hans/.rustup/tmp
total 0
drwxr-xr-x  4 hans  staff  128 Dec 16 14:13 .
drwxr-xr-x  8 hans  staff  256 Dec  1 23:33 ..
drwxr-xr-x  2 hans  staff   64 Nov 30 07:31 bisector-nightly-2021-10-17-aarch64-apple-darwin.JxDhLxyvpW8Z
drwxr-xr-x  2 hans  staff   64 Dec 11 21:49 bisector-nightly-2021-12-01-aarch64-apple-darwin.BU6bdvSEeshX

/Users/hans/.rustup/tmp/bisector-nightly-2021-10-17-aarch64-apple-darwin.JxDhLxyvpW8Z:
total 0
drwxr-xr-x  2 hans  staff   64 Nov 30 07:31 .
drwxr-xr-x  4 hans  staff  128 Dec 16 14:13 ..

/Users/hans/.rustup/tmp/bisector-nightly-2021-12-01-aarch64-apple-darwin.BU6bdvSEeshX:
total 0
drwxr-xr-x  2 hans  staff   64 Dec 11 21:49 .
drwxr-xr-x  4 hans  staff  128 Dec 16 14:13 ..

No obvious permissions problems. I did not try to remove the directories manually yet.

hkratz avatar Dec 16 '21 13:12 hkratz

Hopefully the maintainers of remove_dir_all will be able to resolve this.

kinnison avatar Dec 27 '21 10:12 kinnison

I'm getting this error just after upgrading to rustc v1.58.1.

       stable-x86_64-apple-darwin updated - rustc 1.58.1 (db9d1b20b 2022-01-20) (from rustc 1.58.0 (02072b482 2022-01-11))
  stable-x86_64-unknown-linux-gnu updated - (error reading rustc version) (from (error reading rustc version))
      nightly-x86_64-apple-darwin updated - rustc 1.60.0-nightly (777bb86bc 2022-01-20) (from rustc 1.59.0-nightly (0fb1c371d 2021-12-06))

info: cleaning up downloads & tmp directories
thread 'main' panicked at 'Unable to clean up /Users/myusername/.rustup/tmp: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }', src/utils/utils.rs:642:13

I have write permissions for everything in ~/.rustup/tmp:

❯ exa -lR ~/.rustup/tmp
drwxr-xr-x - myusername  9 Jan 23:22 nm7actibc2gqwju0_dir

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir:
drwxr-xr-x    - myusername  9 Jan 23:22 cargo
.rw-r--r--   40 myusername  9 Jan 23:22 git-commit-hash
.rwxr-xr-x  28k myusername  9 Jan 23:22 install.sh
.rw-r--r--  11k myusername  9 Jan 23:22 LICENSE-APACHE
.rw-r--r-- 1.0k myusername  9 Jan 23:22 LICENSE-MIT
.rw-r--r--  68k myusername  9 Jan 23:22 LICENSE-THIRD-PARTY
.rw-r--r-- 2.7k myusername  9 Jan 23:22 README.md
.rw-r--r--    2 myusername  9 Jan 23:22 rust-installer-version
.rw-r--r--   29 myusername  9 Jan 23:22 version

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir/cargo:
drwxr-xr-x    - myusername  9 Jan 23:22 bin
drwxr-xr-x    - myusername  9 Jan 23:22 etc
drwxr-xr-x    - myusername  9 Jan 23:22 libexec
.rw-r--r-- 1.4k myusername  9 Jan 23:22 manifest.in
drwxr-xr-x    - myusername  9 Jan 23:22 share

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir/cargo/bin:
.rwxr-xr-x 17M myusername  9 Jan 23:22 cargo

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir/cargo/etc:
drwxr-xr-x - myusername  9 Jan 23:22 bash_completion.d

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir/cargo/etc/bash_completion.d:
.rw-r--r-- 9.4k myusername  9 Jan 23:22 cargo

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir/cargo/libexec:
.rwxr-xr-x 3.9M myusername  9 Jan 23:22 cargo-credential-1password

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir/cargo/share:
drwxr-xr-x - myusername  9 Jan 23:22 doc
drwxr-xr-x - myusername  9 Jan 23:22 man
drwxr-xr-x - myusername  9 Jan 23:22 zsh

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir/cargo/share/doc:
drwxr-xr-x - myusername  9 Jan 23:22 cargo

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir/cargo/share/doc/cargo:
.rw-r--r--  11k myusername  9 Jan 23:22 LICENSE-APACHE
.rw-r--r-- 1.0k myusername  9 Jan 23:22 LICENSE-MIT
.rw-r--r--  68k myusername  9 Jan 23:22 LICENSE-THIRD-PARTY
.rw-r--r-- 2.7k myusername  9 Jan 23:22 README.md

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir/cargo/share/man:
drwxr-xr-x - myusername  9 Jan 23:22 man1

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir/cargo/share/man/man1:
.rw-r--r--  16k myusername  9 Jan 23:22 cargo-bench.1
.rw-r--r--  13k myusername  9 Jan 23:22 cargo-build.1
.rw-r--r--  13k myusername  9 Jan 23:22 cargo-check.1
.rw-r--r-- 5.6k myusername  9 Jan 23:22 cargo-clean.1
.rw-r--r--  11k myusername  9 Jan 23:22 cargo-doc.1
.rw-r--r-- 5.0k myusername  9 Jan 23:22 cargo-fetch.1
.rw-r--r--  16k myusername  9 Jan 23:22 cargo-fix.1
.rw-r--r-- 4.0k myusername  9 Jan 23:22 cargo-generate-lockfile.1
.rw-r--r--  487 myusername  9 Jan 23:22 cargo-help.1
.rw-r--r-- 4.1k myusername  9 Jan 23:22 cargo-init.1
.rw-r--r--  13k myusername  9 Jan 23:22 cargo-install.1
.rw-r--r-- 3.0k myusername  9 Jan 23:22 cargo-locate-project.1
.rw-r--r-- 3.1k myusername  9 Jan 23:22 cargo-login.1
.rw-r--r--  18k myusername  9 Jan 23:22 cargo-metadata.1
.rw-r--r-- 3.9k myusername  9 Jan 23:22 cargo-new.1
.rw-r--r-- 4.5k myusername  9 Jan 23:22 cargo-owner.1
.rw-r--r--  10k myusername  9 Jan 23:22 cargo-package.1
.rw-r--r-- 5.6k myusername  9 Jan 23:22 cargo-pkgid.1
.rw-r--r-- 8.7k myusername  9 Jan 23:22 cargo-publish.1
.rw-r--r-- 9.1k myusername  9 Jan 23:22 cargo-run.1
.rw-r--r--  13k myusername  9 Jan 23:22 cargo-rustc.1
.rw-r--r--  12k myusername  9 Jan 23:22 cargo-rustdoc.1
.rw-r--r-- 2.9k myusername  9 Jan 23:22 cargo-search.1
.rw-r--r--  17k myusername  9 Jan 23:22 cargo-test.1
.rw-r--r--  14k myusername  9 Jan 23:22 cargo-tree.1
.rw-r--r-- 3.2k myusername  9 Jan 23:22 cargo-uninstall.1
.rw-r--r-- 5.6k myusername  9 Jan 23:22 cargo-update.1
.rw-r--r-- 5.4k myusername  9 Jan 23:22 cargo-vendor.1
.rw-r--r-- 3.9k myusername  9 Jan 23:22 cargo-verify-project.1
.rw-r--r--  661 myusername  9 Jan 23:22 cargo-version.1
.rw-r--r-- 4.1k myusername  9 Jan 23:22 cargo-yank.1
.rw-r--r-- 8.9k myusername  9 Jan 23:22 cargo.1

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir/cargo/share/zsh:
drwxr-xr-x - myusername  9 Jan 23:22 site-functions

/Users/myusername/.rustup/tmp/nm7actibc2gqwju0_dir/cargo/share/zsh/site-functions:

~~Maybe this is related to the security fix for std::fs::remove_dir_all?~~ I didn't realise the removing of the temporary directory was handled by another crate, see https://github.com/rust-lang/rustup/issues/2895#issuecomment-1018471766

cherryblossom000 avatar Jan 21 '22 08:01 cherryblossom000

@cherryblossom000 I don't think its related; the remove_dir_all crate is not the same code as in the stdlib.

rbtcollins avatar Jan 21 '22 12:01 rbtcollins

@rbtcollins Merging https://github.com/XAMPPRocky/remove_dir_all/pull/36 should fix this particular issue for Mac and other BSDs.

hkratz avatar Jan 21 '22 12:01 hkratz

I was having this issue too but it seems to be resolved when I stopped a cargo watch -x 'c' that was running on a different terminal. IDK how Windows work but maybe because another process is using it?

hbina avatar May 25 '22 03:05 hbina

I am running into this error today:

$ RUST_BACKTRACE=1 rustup update
info: syncing channel updates for 'stable-x86_64-apple-darwin'
info: syncing channel updates for 'beta-x86_64-apple-darwin'
info: syncing channel updates for 'nightly-x86_64-apple-darwin'

   stable-x86_64-apple-darwin unchanged - rustc 1.65.0 (897e37553 2022-11-02)
     beta-x86_64-apple-darwin unchanged - rustc 1.66.0-beta.1 (e080cc5a6 2022-11-01)
  nightly-x86_64-apple-darwin unchanged - rustc 1.67.0-nightly (09508489e 2022-11-04)

info: cleaning up downloads & tmp directories
thread 'main' panicked at 'Unable to clean up /Users/lopopolo/.rustup/tmp: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }', src/utils/utils.rs:649:13
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: rustup::utils::utils::delete_dir_contents
   3: rustup::cli::rustup_mode::update
   4: rustup::cli::rustup_mode::main
   5: rustup_init::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

It looks like maybe there's a toolchain in there from cargo-bisect-rustc?

$ tree /Users/lopopolo/.rustup/tmp
/Users/lopopolo/.rustup/tmp
└── bisector-nightly-2022-08-19-x86_64-apple-darwinr9RXpH
    ├── bin
    │   ├── rust-gdb
    │   └── rust-lldb
    ├── lib
    │   ├── librustc_driver-6a2677ff84a3c96b.dylib
    │   ├── libstd-c047ccf37d2c9989.dylib
    │   ├── libtest-a4cc331024c39a98.dylib
    │   └── rustlib
    │       ├── etc
    │       └── x86_64-apple-darwin
    │           ├── bin
    │           │   └── gcc-ld
    │           └── codegen-backends
    ├── libexec
    └── share
        ├── doc
        │   └── rust
        │       ├── COPYRIGHT
        │       ├── LICENSE-APACHE
        │       └── LICENSE-MIT
        └── man
            └── man1
                ├── rustc.1
                └── rustdoc.1

15 directories, 10 files

Meta

$ sw_vers
ProductName:    macOS
ProductVersion: 12.6
BuildVersion:   21G115
$ rustup -Vv
rustup 1.25.1 (2022-07-12)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active `rustc` version is `rustc 1.65.0 (897e37553 2022-11-02)`

lopopolo avatar Nov 05 '22 14:11 lopopolo

got this issue today too

olivertzeng avatar Nov 22 '22 11:11 olivertzeng

Same for me on every rustup update. Fwiw, I set Rustup home to inside $XDG_DATA_HOME, if that effects things (though it shouldn't?):

❯ RUST_BACKTRACE=full rustup update
info: syncing channel updates for 'stable-aarch64-apple-darwin'
info: syncing channel updates for 'nightly-aarch64-apple-darwin'

   stable-aarch64-apple-darwin unchanged - rustc 1.65.0 (897e37553 2022-11-02)
  nightly-aarch64-apple-darwin unchanged - rustc 1.67.0-nightly (32e613bba 2022-12-02)

info: cleaning up downloads & tmp directories
thread 'main' panicked at 'Unable to clean up /Users/bene/.local/share/rustup/tmp: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }', src/utils/utils.rs:649:13
stack backtrace:
   0:        0x1050aa668 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::ha610dc96df5533b5
   1:        0x104d16f88 - core::fmt::write::h85a0222d7173f527
   2:        0x105090f2c - std::io::Write::write_fmt::h1e81e1aae1f60926
   3:        0x10509c69c - std::panicking::default_hook::{{closure}}::hafcdd34daa6aee47
   4:        0x10509c39c - std::panicking::default_hook::hbdef0c99978ce86d
   5:        0x10509cff8 - std::panicking::rust_panic_with_hook::he857adbde651a96b
   6:        0x1050aac9c - std::panicking::begin_panic_handler::{{closure}}::h78d4b95de3b3c015
   7:        0x1050aac14 - std::sys_common::backtrace::__rust_end_short_backtrace::h948efd9d51b5e877
   8:        0x10509cab4 - _rust_begin_unwind
   9:        0x10514472c - core::panicking::panic_fmt::h4c391ad4ab25a9e0
  10:        0x104fb0dcc - rustup::utils::utils::delete_dir_contents::hd739cc10628ca647
  11:        0x104f93180 - rustup::cli::rustup_mode::update::hab527212ec6417db
  12:        0x104f7d2f0 - rustup::cli::rustup_mode::main::hc907663a8a5d3a35
  13:        0x104ccc09c - rustup_init::main::h08e08969caf302d0
  14:        0x104cca24c - std::sys_common::backtrace::__rust_begin_short_backtrace::he6ec1093de90141b
  15:        0x104ccc76c - _main


########### info ##########
❯ lsd -al ~/.local/share/rustup/tmp/
drwxr-xr-x bene staff 192 B 2 minutes ago  .
drwxr-xr-x bene staff 256 B 7 months ago   ..
drwxr-xr-x bene staff 384 B 6 months ago   m1hsw6y9gclxm5ki_dir
.rw-r--r-- bene staff  27 B 6 months ago   1t3wvduj6a3436gd_file
.rw-r--r-- bene staff  63 B 6 months ago   lr6unlwf9mj_hf1b_file
.rw-r--r-- bene staff   1 B 6 months ago   x3_k1cl1471rrwyv_file

❯ rustup show
Default host: aarch64-apple-darwin
rustup home:  /Users/bene/.local/share/rustup

installed toolchains
--------------------

stable-aarch64-apple-darwin (default)
nightly-aarch64-apple-darwin
1.61.0-aarch64-apple-darwin

active toolchain
----------------

stable-aarch64-apple-darwin (default)
rustc 1.65.0 (897e37553 2022-11-02)

❯ rustup --version
rustup 1.25.1 (2022-07-12)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active `rustc` version is `rustc 1.65.0 (897e37553 2022-11-02)`

❯ sw_vers 
ProductName:		macOS
ProductVersion:		13.0.1
BuildVersion:		22A400

❯ uname -mprsv
Darwin 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct  9 20:15:09 PDT 2022; root:xnu-8792.41.9~2/RELEASE_ARM64_T6000 arm64 arm

❯ rg rust ~/.config/fish/
/Users/bene/.config/fish/conf.d/xdg_env.fish
27:# Rust
29:set -gx RUSTUP_HOME $XDG_DATA_HOME/rustup

/Users/bene/.config/fish/completions/rustup.fish
1:# Generated by `rustup completions fish`
...

diktomat avatar Dec 03 '22 15:12 diktomat

got this issue today too

MohamedBsh avatar Dec 04 '22 15:12 MohamedBsh