rust
rust copied to clipboard
rust-lang
rustc panics due to stale incremental compilation info
Summary
Unfortunately I cannot reproduce this after running cargo clean.
I was implementing the Not trait rather than implementing .not() directly and at some point I got this panick when running clippy. cargo build still worked fine.
Here's the branch I was working on: https://github.com/pola-rs/polars/tree/72aa9515f385f1a2d30802a50ef851fcf92c2e6a
I hope this may still be useful somehow.
Version
rustc 1.77.0-nightly (d6d7a9386 2023-12-22)
binary: rustc
commit-hash: d6d7a93866f2ffcfb51828b8859bdad760b54ce0
commit-date: 2023-12-22
host: x86_64-unknown-linux-gnu
release: 1.77.0-nightly
LLVM version: 17.0.6
Error output
Backtrace
cargo clippy --workspace --all-targets --all-features --locked -- -D warnings
Checking polars-ops v0.36.2 (/home/stijn/code/polars/crates/polars-ops)
thread 'rustc' panicked at library/alloc/src/raw_vec.rs:571:5:
capacity overflow
stack backtrace:
thread 'rustc' panicked at library/alloc/src/raw_vec.rs:571:5:
capacity overflow
stack backtrace:
0: 0x7ff54598b6f6 - std::backtrace_rs::backtrace::libunwind::trace::h4c39d39dfbe9783d
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/../../backtrace/src/backtrace/libunwind.rs:104:5
1: 0x7ff54598b6f6 - std::backtrace_rs::backtrace::trace_unsynchronized::h4dc8ace54ae311e5
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x7ff54598b6f6 - std::sys_common::backtrace::_print_fmt::hb93bb30078f88f5e
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/sys_common/backtrace.rs:68:5
3: 0x7ff54598b6f6 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hc0a088b7e0825f6a
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/sys_common/backtrace.rs:44:22
4: 0x7ff5459ddf40 - core::fmt::rt::Argument::fmt::h3bf37701eb7e7bdb
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/core/src/fmt/rt.rs:142:9
5: 0x7ff5459ddf40 - core::fmt::write::h7e6ab5f6daf5c422
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/core/src/fmt/mod.rs:1120:17
6: 0x7ff54597f4af - std::io::Write::write_fmt::h08c3306cb5bf723d
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/io/mod.rs:1810:15
7: 0x7ff54598b4d4 - std::sys_common::backtrace::_print::hafed43d2fa285a46
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/sys_common/backtrace.rs:47:5
8: 0x7ff54598b4d4 - std::sys_common::backtrace::print::h39c23a58558ca2e2
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/sys_common/backtrace.rs:34:9
9: 0x7ff54598e267 - std::panicking::default_hook::{{closure}}::h33423deafbc8e615
10: 0x7ff54598dfc9 - std::panicking::default_hook::h50ae27bf0009ee3b
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/panicking.rs:292:9
0: 0x7f0df2d8b6f6 - std::backtrace_rs::backtrace::libunwind::trace::h4c39d39dfbe9783d
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/../../backtrace/src/backtrace/libunwind.rs:104:5
1: 0x7f0df2d8b6f6 - std::backtrace_rs::backtrace::trace_unsynchronized::h4dc8ace54ae311e5
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x7f0df2d8b6f6 - std::sys_common::backtrace::_print_fmt::hb93bb30078f88f5e
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/sys_common/backtrace.rs:68:5
3: 0x7f0df2d8b6f6 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hc0a088b7e0825f6a
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/sys_common/backtrace.rs:44:22
4: 0x7f0df2dddf40 - core::fmt::rt::Argument::fmt::h3bf37701eb7e7bdb
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/core/src/fmt/rt.rs:142:9
5: 0x7f0df2dddf40 - core::fmt::write::h7e6ab5f6daf5c422
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/core/src/fmt/mod.rs:1120:17
6: 0x7f0df2d7f4af - std::io::Write::write_fmt::h08c3306cb5bf723d
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/io/mod.rs:1810:15
7: 0x7f0df2d8b4d4 - std::sys_common::backtrace::_print::hafed43d2fa285a46
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/sys_common/backtrace.rs:47:5
8: 0x7f0df2d8b4d4 - std::sys_common::backtrace::print::h39c23a58558ca2e2
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/sys_common/backtrace.rs:34:9
9: 0x7f0df2d8e267 - std::panicking::default_hook::{{closure}}::h33423deafbc8e615
10: 0x7f0df2d8dfc9 - std::panicking::default_hook::h50ae27bf0009ee3b
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/panicking.rs:292:9
11: 0x7ff548782ccc - std[3a009c5aa3cd6e18]::panicking::update_hook::<alloc[7046a3554d0212f6]::boxed::Box<rustc_driver_impl[99e8362fbaa298ef]::install_ice_hook::{closure#0}>>::{closure#0}
12: 0x7ff54598e9b6 - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h5ddf83902817cab1
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/alloc/src/boxed.rs:2029:9
13: 0x7ff54598e9b6 - std::panicking::rust_panic_with_hook::hbc586ac9df6b73a8
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/panicking.rs:783:13
14: 0x7ff54598e6c9 - std::panicking::begin_panic_handler::{{closure}}::hf311b40154e75f12
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/panicking.rs:649:13
15: 0x7ff54598bbf6 - std::sys_common::backtrace::__rust_end_short_backtrace::he9e1eb231bad02f7
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/sys_common/backtrace.rs:171:18
16: 0x7ff54598e460 - rust_begin_unwind
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/panicking.rs:645:5
17: 0x7ff5459da645 - core::panicking::panic_fmt::hefa1490703b8b929
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/core/src/panicking.rs:72:14
18: 0x7ff5459cb98f - alloc::raw_vec::capacity_overflow::h0a679f9c4fe31dca
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/alloc/src/raw_vec.rs:571:5
19: 0x7ff54aac931c - rustc_incremental[c3ec564ce92581fa]::persist::load::setup_dep_graph
20: 0x7ff54a836598 - <rustc_interface[dfee6d40a3df20bb]::queries::Queries>::global_ctxt
21: 0x7ff54a99f082 - rustc_interface[dfee6d40a3df20bb]::interface::run_compiler::<core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>, rustc_driver_impl[99e8362fbaa298ef]::run_compiler::{closure#0}>::{closure#0}
22: 0x7ff54a925346 - std[3a009c5aa3cd6e18]::sys_common::backtrace::__rust_begin_short_backtrace::<rustc_interface[dfee6d40a3df20bb]::util::run_in_thread_with_globals<rustc_interface[dfee6d40a3df20bb]::util::run_in_thread_pool_with_globals<rustc_interface[dfee6d40a3df20bb]::interface::run_compiler<core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>, rustc_driver_impl[99e8362fbaa298ef]::run_compiler::{closure#0}>::{closure#0}, core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>>::{closure#0}, core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>>
23: 0x7ff54a925173 - <<std[3a009c5aa3cd6e18]::thread::Builder>::spawn_unchecked_<rustc_interface[dfee6d40a3df20bb]::util::run_in_thread_with_globals<rustc_interface[dfee6d40a3df20bb]::util::run_in_thread_pool_with_globals<rustc_interface[dfee6d40a3df20bb]::interface::run_compiler<core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>, rustc_driver_impl[99e8362fbaa298ef]::run_compiler::{closure#0}>::{closure#0}, core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>>::{closure#0}, core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>>::{closure#1} as core[2b984803ee3f5e21]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
24: 0x7ff5459988e5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h446fa4c55b1bf8c4
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/alloc/src/boxed.rs:2015:9
25: 0x7ff5459988e5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hb661ed53115c5033
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/alloc/src/boxed.rs:2015:9
26: 0x7ff5459988e5 - std::sys::unix::thread::Thread::new::thread_start::hbd74c47d36330101
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/sys/unix/thread.rs:108:17
27: 0x7ff54565fac3 - <unknown>
28: 0x7ff5456f1a40 - <unknown>
29: 0x0 - <unknown>
11: 0x7f0df5b82ccc - std[3a009c5aa3cd6e18]::panicking::update_hook::<alloc[7046a3554d0212f6]::boxed::Box<rustc_driver_impl[99e8362fbaa298ef]::install_ice_hook::{closure#0}>>::{closure#0}
12: 0x7f0df2d8e9b6 - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h5ddf83902817cab1
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/alloc/src/boxed.rs:2029:9
13: 0x7f0df2d8e9b6 - std::panicking::rust_panic_with_hook::hbc586ac9df6b73a8
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/panicking.rs:783:13
14: 0x7f0df2d8e6c9 - std::panicking::begin_panic_handler::{{closure}}::hf311b40154e75f12
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/panicking.rs:649:13
15: 0x7f0df2d8bbf6 - std::sys_common::backtrace::__rust_end_short_backtrace::he9e1eb231bad02f7
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/sys_common/backtrace.rs:171:18
16: 0x7f0df2d8e460 - rust_begin_unwind
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/panicking.rs:645:5
17: 0x7f0df2dda645 - core::panicking::panic_fmt::hefa1490703b8b929
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/core/src/panicking.rs:72:14
18: 0x7f0df2dcb98f - alloc::raw_vec::capacity_overflow::h0a679f9c4fe31dca
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/alloc/src/raw_vec.rs:571:5
19: 0x7f0df7ec931c - rustc_incremental[c3ec564ce92581fa]::persist::load::setup_dep_graph
20: 0x7f0df7c36598 - <rustc_interface[dfee6d40a3df20bb]::queries::Queries>::global_ctxt
21: 0x7f0df7d9f082 - rustc_interface[dfee6d40a3df20bb]::interface::run_compiler::<core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>, rustc_driver_impl[99e8362fbaa298ef]::run_compiler::{closure#0}>::{closure#0}
22: 0x7f0df7d25346 - std[3a009c5aa3cd6e18]::sys_common::backtrace::__rust_begin_short_backtrace::<rustc_interface[dfee6d40a3df20bb]::util::run_in_thread_with_globals<rustc_interface[dfee6d40a3df20bb]::util::run_in_thread_pool_with_globals<rustc_interface[dfee6d40a3df20bb]::interface::run_compiler<core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>, rustc_driver_impl[99e8362fbaa298ef]::run_compiler::{closure#0}>::{closure#0}, core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>>::{closure#0}, core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>>
23: 0x7f0df7d25173 - <<std[3a009c5aa3cd6e18]::thread::Builder>::spawn_unchecked_<rustc_interface[dfee6d40a3df20bb]::util::run_in_thread_with_globals<rustc_interface[dfee6d40a3df20bb]::util::run_in_thread_pool_with_globals<rustc_interface[dfee6d40a3df20bb]::interface::run_compiler<core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>, rustc_driver_impl[99e8362fbaa298ef]::run_compiler::{closure#0}>::{closure#0}, core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>>::{closure#0}, core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[2b984803ee3f5e21]::result::Result<(), rustc_span[fa12e73b1270738]::ErrorGuaranteed>>::{closure#1} as core[2b984803ee3f5e21]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
24: 0x7f0df2d988e5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h446fa4c55b1bf8c4
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/alloc/src/boxed.rs:2015:9
25: 0x7f0df2d988e5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hb661ed53115c5033
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/alloc/src/boxed.rs:2015:9
26: 0x7f0df2d988e5 - std::sys::unix::thread::Thread::new::thread_start::hbd74c47d36330101
at /rustc/d6d7a93866f2ffcfb51828b8859bdad760b54ce0/library/std/src/sys/unix/thread.rs:108:17
27: 0x7f0df2a5fac3 - <unknown>
28: 0x7f0df2af1a40 - <unknown>
29: 0x0 - <unknown>
error: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust-clippy/issues/new?template=ice.yml
note: please attach the file at `/home/stijn/code/polars/rustc-ice-2024-01-17T09_42_36-19672.txt` to your bug report
note: compiler flags: --crate-type lib -C embed-bitcode=no -C debuginfo=2 -C incremental=[REDACTED]
note: some of the compiler flags provided by cargo are hidden
query stack during panic:
end of query stack
note: Clippy version: clippy 0.1.76 (d6d7a93 2023-12-22)
error: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust-clippy/issues/new?template=ice.yml
note: please attach the file at `/home/stijn/code/polars/rustc-ice-2024-01-17T09_42_36-19674.txt` to your bug report
note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental=[REDACTED]
note: some of the compiler flags provided by cargo are hidden
query stack during panic:
end of query stack
note: Clippy version: clippy 0.1.76 (d6d7a93 2023-12-22)
error: could not compile `polars-ops` (lib)
warning: build failed, waiting for other jobs to finish...
error: could not compile `polars-ops` (lib test)
make: *** [Makefile:81: clippy] Error 101
Hi thanks for the report!
At a first glance this doesn't seem like a clippy issue but a compiler internals issue. I will take a look, if it is, I will open an issue in rustc, otherwise will submit a PR.
@rustbot claim
Unfortunately I cannot reproduce this after running cargo clean.
Maybe some stale incremental compilation state, that caused rustc to panic? This indeed doesn't sound like a Clippy issue. Curious what you come up with and how to reproduce this.
I pulled that specific commit, and couldn't reproduce with the nightly-2024-01-11 toolchain.
It seems that the issue may have happened during compilation as @flip1995 suggested, and not during clippy run.
I run this in the repo dir:
cargo +nightly-2024-01-11 clippy
I will try going through the commit history.
Maybe some stale incremental compilation state, that caused rustc to panic?
That's what I am thinking. At some point I ran into the panic, then I was reverting changes but it kept happening, and then after running cargo clean the panic went away. And I could re-apply my changes without running into the panic again.
Unfortunately that made it impossible for me to reliably reproduce, given my limited knowledge of this subject.
What I am thinking is that somehow cargo build had access to previous files that resulted in a successful compilation, and running clippy caused a rebuild with a stale state.
From the stacktrace it appears that the issue is related to the query engine ie type checking, and the compilation error I am getting is related to that as well 🤔.
It looks unlikely, but if you run something like bacon it's more probable.
Accessing deleted files by having references to them is a thing that can occur in linux systems.
This example comes to mind https://lkml.iu.edu/hypermail/linux/kernel/2401.3/05626.html
Should we close this issue or move it to rustc? As there doesn't seem to be a reproducer, it's almost impossible to fix this.
@saethlin This is unfortunately beyond my skills at the moment. Please take it away if you want.
The crash here is happening when loading the incremental compilation dependency graph. I've worked on that code a fair bit.
What jumps out at me is that we blindly assume that the last 16 bytes are the number of edges and number of nodes. Usually when decoders crash with a capacity overflow or allocation error it's because they're trying to pre-allocate for an array, and the decoder read some random bytes instead of a length. The encoding format tries to adapt itself to minimize the number of zero bytes it writes, so the odds are actually quite good that if you grab a random 8 bytes of the dep graph file and then try to convert it from an element count to a number of bytes for an array's whole allocation (as the code in RawVec that crashes does), that multiplication will overflow.
So, it seems likely to me that somehow your dep graph file was incomplete. I've been trying to fix a situation that looks a whole lot like this, where if you ran out of disk space during a compile, rustc would fail to report an error correctly and just tell Cargo it finished compilation. Then a new rustc process would start up and try to load a truncated file and crash.
I'm afraid this looks exactly like that. @stinodego is there any chance you ran out of disk space recently? From the rustc version you reported, it looks like you're on a nightly before https://github.com/rust-lang/rust/pull/119510 landed.
If there's no way you ran out of disk space recently, I have a patch that would at least let us detect that there's a truncated file and crash more helpfully.
I guess it's possible - I run on WSL and the virtual disk space gets filled up sometimes.
w/r/t the Serenity example - disk space isn't an issue in my case, I can go check rn but my disk should be like 90-95% unused
I think the root cause of this issue was rustc incorrectly handling/reporting I/O errors. I've landed a number of PRs to address the problem; the most recent of those is https://github.com/rust-lang/rust/pull/119510 which is now part of the latest stable toolchain, 1.77. Therefore, even though I have not reproduced exactly what you've reported here, I'm going to close this because I am reasonably confident that this bug is now fixed on stable.
Of course it is possible that I am wrong and the bug you've reported here is not fixed. If you happen to run into problems like this in 1.77 or later please do not hesitate to open a new issue.