procfs icon indicating copy to clipboard operation
procfs copied to clipboard

Surprisingly large compile time

Open saethlin opened this issue 3 years ago • 1 comments

In a decently large project with 255 dependencies, this crate ranks 6th by compile time in a debug build:

crate build time
protobuf v2.18.1 5.2s
syn v1.0.58 5.2s
regex-syntax v0.6.18 4.0s
nix v0.18.0 3.5s
object v0.20.0 3.2s
clap v2.33.3 3.2s
procfs v0.9.1 3.0s
gimli v0.22.0 3.0s
serde_derive v1.0.116 2.8s
trust-dns-proto v0.20.1 2.6s

It looks like approximately 40% of this crate's build time was introduced by the strategy employed in https://github.com/eminence/procfs/pull/48. As far as I can understand, the idea there was to nest macros so that the value produced line!() is relevant to the error. The problem is that this strategy causes this crate to balloon after macro expansion. If I run cargo llvm-lines | head in this crate, I get this:

  Lines          Copies       Function name
  -----          ------       -------------
  250761 (100%)  5134 (100%)  (TOTAL)
   10987 (4.4%)     1 (0.0%)  procfs::process::status::Status::from_reader
    7804 (3.1%)     1 (0.0%)  procfs::process::mount::NFSEventCounter::from_str
    5989 (2.4%)     1 (0.0%)  procfs::process::stat::Stat::from_reader
    3895 (1.6%)     1 (0.0%)  procfs::diskstats::DiskStat::from_line
    3875 (1.5%)    25 (0.5%)  alloc::raw_vec::RawVec<T,A>::grow_amortized
    3698 (1.5%)    76 (1.5%)  <core::result::Result<T,E> as core::ops::try_trait::Try>::branch
    3018 (1.2%)     1 (0.0%)  procfs::process::limit::Limits::from_reader

which is quite unusual, and points squarely at the large amount of code that (for example) from_str! expands into.

I'm also not convinced this strategy even produces useful errors, because its error messages are rendered useless by helper functions, such as from_iter and split_into_num.

Are you interested in a PR that doesn't introduce the panics back, but which tries to do something about the compile time regression from #48, even if it removes some of the file+line information?

saethlin avatar Jun 05 '21 23:06 saethlin

Hi there,

I for sure do not want the panics, but I would certainly welcome ideas about strategies that can improve compile times.

I'm also not convinced this strategy even produces useful errors, because its error messages are rendered useless by helper functions, such as from_iter and split_into_num.

Could you say a bit more about this, or maybe provide an example? I have to admit, the error messages mostly designed for me, so if I receive a bug report, I can pretty quickly see where the parsing problem is. But many of them haven't actually been triggered (and so there could easily be some poorly constructed error messages).

Here's an example, though, that I came up with quickly (I just added this to the bottom of net.rs):

#[test]
fn test_error_msg() {
    let _ = DeviceStatus::from_str("eth0 -1").unwrap();
}

And the resulting error message is:

---- net::tests::test_error_msg stdout ----
thread 'net::tests::test_error_msg' panicked at 'called `Result::unwrap()` on an `Err` value: InternalError(bug at src/net.rs:573 (please report this procfs bug)        
Internal Unwrap Error: Internal error: bug at src/lib.rs:260 (please report this procfs bug)                                                                             
Internal Unwrap Error: Failed to convert)', src/net.rs:727:51
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The useful pointer for me is the bug at src/net.rs:573 part. I admit the pointers to lib.rs:260 and net.rs:727 are pretty useless, though.

I guess one possibility would be to have this type of error message (capturing the exact line number) only in debug builds, not release builds, but maybe that doesn't really get rid of the pain.

eminence avatar Jun 07 '21 02:06 eminence