rust-clippy
rust-clippy copied to clipboard
rust-lang
Most time consuming test files
Recently, I've run an experiment using SPEEDTEST of, benchmarking each test file 100 times and getting the average time, then recollecting all that data into a single file containing average of milliseconds + number of lines in that test file (+ filename), and using a Python script (had to de-rust here, I can now see why people use Python for data science hehe :3)
So, now we have a list of the top 150 most consuming lints out there.
| Milliseconds | Lines | File |
|---|---|---|
| 108 | 11 | tests/ui/print_stderr.rs |
| 1012 | 103 | tests/ui/default_trait_access.rs |
| 1209 | 122 | tests/ui/shadow.rs |
| 825 | 82 | tests/ui/mut_mut.rs |
| 930 | 92 | tests/ui/almost_complete_range.rs |
| 912 | 89 | tests/ui/redundant_field_names.rs |
| 902 | 88 | tests/ui/min_ident_chars.rs |
| 205 | 20 | tests/ui/unnested_or_patterns2.rs |
| 205 | 20 | tests/ui/vec_resize_to_zero.rs |
| 205 | 20 | tests/ui/default_instead_of_iter_empty.rs |
| 206 | 20 | tests/ui/tabs_in_doc_comments.rs |
| 207 | 20 | tests/ui/ptr_offset_with_cast.rs |
| 306 | 29 | tests/ui/reversed_empty_ranges_fixable.rs |
| 203 | 19 | tests/ui/tests_outside_test_module.rs |
| 107 | 10 | tests/ui/double_neg.rs |
| 312 | 29 | tests/ui/skip_while_next.rs |
| 410 | 38 | tests/ui/iter_skip_next.rs |
| 206 | 19 | tests/ui/reversed_empty_ranges_unfixable.rs |
| 206 | 19 | tests/ui/string_add_assign.rs |
| 206 | 19 | tests/ui/zero_ptr_no_std.rs |
| 207 | 19 | tests/ui/floating_point_exp.rs |
| 1018 | 91 | tests/ui/equatable_if_let.rs |
| 213 | 19 | tests/ui/filter_map_next.rs |
| 852 | 75 | tests/ui/single_call_fn.rs |
| 205 | 18 | tests/ui/inline_fn_without_body.rs |
| 205 | 18 | tests/ui/writeln_empty_string.rs |
| 915 | 80 | tests/ui/manual_rem_euclid.rs |
| 206 | 18 | tests/ui/borrow_as_ptr.rs |
| 288 | 25 | tests/ui/bind_instead_of_map.rs |
| 105 | 9 | tests/ui/deprecated_old.rs |
| 247 | 21 | tests/ui/filter_map_next_fixable.rs |
| 106 | 9 | tests/ui/str_to_string.rs |
| 107 | 9 | tests/ui/builtin_type_shadow.rs |
| 1980 | 166 | tests/ui/format_args.rs |
| 920 | 77 | tests/ui/inconsistent_struct_constructor.rs |
| 205 | 17 | tests/ui/println_empty_string.rs |
| 956 | 79 | tests/ui/needless_return_with_question_mark.rs |
| 209 | 17 | tests/ui/floating_point_logbase.rs |
| 911 | 74 | tests/ui/iter_over_hash_type.rs |
| 137 | 11 | tests/ui/if_not_else_bittest.rs |
| 918 | 73 | tests/ui/borrow_deref_ref.rs |
| 954 | 75 | tests/ui/let_underscore_untyped.rs |
| 217 | 17 | tests/ui/filter_map_identity.rs |
| 205 | 16 | tests/ui/entry_btree.rs |
| 205 | 16 | tests/ui/is_digit_ascii_radix.rs |
| 205 | 16 | tests/ui/unchecked_duration_subtraction.rs |
| 206 | 16 | tests/ui/unknown_clippy_lints.rs |
| 206 | 16 | tests/ui/unused_rounding.rs |
| 207 | 16 | tests/ui/short_circuit_statement.rs |
| 104 | 8 | tests/ui/non_minimal_cfg2.rs |
| 406 | 31 | tests/ui/uninlined_format_args_panic.rs |
| 106 | 8 | tests/ui/exit3.rs |
| 214 | 16 | tests/ui/create_dir.rs |
| 108 | 8 | tests/ui/string_to_string.rs |
| 405 | 30 | tests/ui/thread_local_initializer_can_be_made_const.rs |
| 205 | 15 | tests/ui/int_plus_one.rs |
| 205 | 15 | tests/ui/len_zero_ranges.rs |
| 205 | 15 | tests/ui/reversed_empty_ranges_loops_unfixable.rs |
| 206 | 15 | tests/ui/map_collect_result_unit.rs |
| 206 | 15 | tests/ui/repeat_once.rs |
| 207 | 15 | tests/ui/flat_map_identity.rs |
| 207 | 15 | tests/ui/transmute_int_to_char.rs |
| 838 | 59 | tests/ui/single_range_in_vec_init.rs |
| 1213 | 85 | tests/ui/useless_attribute.rs |
| 229 | 16 | tests/ui/iter_next_loop.rs |
| 960 | 67 | tests/ui/needless_arbitrary_self_type.rs |
| 1195 | 81 | tests/ui/filter_map_bool_then.rs |
| 828 | 56 | tests/ui/manual_float_methods.rs |
| 224 | 15 | tests/ui/non_minimal_cfg.rs |
| 912 | 61 | tests/ui/ptr_cast_constness.rs |
| 105 | 7 | tests/ui/empty_enum_without_never_type.rs |
| 1108 | 73 | tests/ui/std_instead_of_core.rs |
| 914 | 59 | tests/ui/deref_addrof.rs |
| 1198 | 76 | tests/ui/literals.rs |
| 205 | 13 | tests/ui/floating_point_hypot.rs |
| 206 | 13 | tests/ui/needless_parens_on_range_literals.rs |
| 206 | 13 | tests/ui/suspicious_command_arg_space.rs |
| 206 | 13 | tests/ui/trailing_zeros.rs |
| 206 | 13 | tests/ui/zero_ptr.rs |
| 209 | 13 | tests/ui/needless_option_take.rs |
| 561 | 34 | tests/ui/find_map.rs |
| 216 | 13 | tests/ui/types.rs |
| 1419 | 85 | tests/ui/manual_assert.rs |
| 875 | 51 | tests/ui/empty_loop.rs |
| 206 | 12 | tests/ui/flat_map_option.rs |
| 920 | 53 | tests/ui/unneeded_wildcard_pattern.rs |
| 838 | 48 | tests/ui/arc_with_non_send_sync.rs |
| 1258 | 71 | tests/ui/impl.rs |
| 911 | 51 | tests/ui/reserve_after_initialization.rs |
| 890 | 49 | tests/ui/allow_attributes.rs |
| 820 | 44 | tests/ui/allow_attributes_without_reason.rs |
| 205 | 11 | tests/ui/track-diagnostics.rs |
| 914 | 49 | tests/ui/string_lit_chars_any.rs |
| 909 | 48 | tests/ui/mistyped_literal_suffix.rs |
| 211 | 11 | tests/ui/as_underscore.rs |
| 826 | 43 | tests/ui/unnecessary_fallible_conversions_unfixable.rs |
| 511 | 26 | tests/ui/manual_find.rs |
| 914 | 45 | tests/ui/manual_slice_size_calculation.rs |
| 204 | 10 | tests/ui/methods_fixable.rs |
| 874 | 42 | tests/ui/let_with_type_underscore.rs |
| 105 | 5 | tests/ui/missing_doc_crate_missing.rs |
| 105 | 5 | tests/ui/missing_inline_executable.rs |
| 993 | 46 | tests/ui/needless_arbitrary_self_type_unfixable.rs |
| 1037 | 48 | tests/ui/four_forward_slashes.rs |
| 1263 | 56 | tests/ui/ignored_unit_patterns.rs |
| 204 | 9 | tests/ui/unnecessary_self_imports.rs |
| 205 | 9 | tests/ui/to_digit_is_some.rs |
| 1162 | 51 | tests/ui/macro_use_imports_expect.rs |
| 206 | 9 | tests/ui/bytes_nth.rs |
| 921 | 39 | tests/ui/toplevel_ref_arg.rs |
| 900 | 38 | tests/ui/pub_with_shorthand.rs |
| 809 | 34 | tests/ui/unneeded_field_pattern.rs |
| 906 | 38 | tests/ui/pub_without_shorthand.rs |
| 1135 | 47 | tests/ui/patterns.rs |
| 2344 | 92 | tests/ui/format.rs |
| 205 | 8 | tests/ui/char_lit_as_u8_suggestions.rs |
| 206 | 8 | tests/ui/empty_enum.rs |
| 207 | 8 | tests/ui/to_string_in_format_args_incremental.rs |
| 208 | 8 | tests/ui/char_lit_as_u8.rs |
| 864 | 33 | tests/ui/needless_pub_self.rs |
| 1261 | 48 | tests/ui/macro_use_imports.rs |
| 106 | 4 | tests/ui/missing_doc_crate.rs |
| 107 | 4 | tests/ui/unknown_attribute.rs |
| 1110 | 41 | tests/ui/unseparated_prefix_literals.rs |
| 824 | 29 | tests/ui/string_add.rs |
| 205 | 7 | tests/ui/open_options_fixable.rs |
| 205 | 7 | tests/ui/path_buf_push_overwrite.rs |
| 326 | 11 | tests/ui/map_unit_fn.rs |
| 208 | 7 | tests/ui/option_map_or_err_ok.rs |
| 812 | 27 | tests/ui/toplevel_ref_arg_non_rustfix.rs |
| 850 | 28 | tests/ui/must_use_unit.rs |
| 403 | 13 | tests/ui/unwrap_or.rs |
| 817 | 26 | tests/ui/proc_macro.rs |
| 205 | 6 | tests/ui/four_forward_slashes_first_line.rs |
| 911 | 26 | tests/ui/redundant_at_rest_pattern.rs |
| 809 | 23 | tests/ui/as_conversions.rs |
| 106 | 3 | tests/ui/issue-3145.rs |
| 951 | 24 | tests/ui/iter_skip_zero.rs |
| 205 | 5 | tests/ui/obfuscated_if_else.rs |
| 211 | 5 | tests/ui/string_from_utf8_as_bytes.rs |
| 887 | 21 | tests/ui/unwrap.rs |
| 828 | 17 | tests/ui/option_env_unwrap.rs |
| 210 | 4 | tests/ui/author.rs |
| 1010 | 19 | tests/ui/range.rs |
| 1137 | 21 | tests/ui/expect.rs |
| 813 | 13 | tests/ui/deref_addrof_macro.rs |
| 809 | 12 | tests/ui/mem_replace_macro.rs |
| 3431 | 35 | tests/ui/large_const_arrays.rs |
| 205 | 2 | tests/ui/renamed_builtin_attr.rs |
| 827 | 6 | tests/ui/unnecessary_fallible_conversions.rs |
I'll do some inspection in these lints (primarily seeing which function calls are repeated between them) and then... Fix them (?) If you want to optimize some of these lints, you're free to do so! (we probably won't collide in the same lint hehe :3)
I've been looking at single_call_fn specifically. It's doing another pass through the whole crate at the very end which doesn't seem necessary. We should be able to do what the visitor is doing at the same time as the lint (noticed it while working on #12183)
@blyxyas Could you explain the table in a bit more detail? Your description says:
benchmarking each lint 100 times
but you list individual test files instead of lints. Do you mean that you ran Clippy 100x on each file and measured that? In that case, it's likely that the time doesn't just measure the tested lint but also other lints, which might be allowed or still got relatively far in their checks.
Maybe related sidenote: For our Metadata Collection Monster :dragon: we actually have a thing, that only registers that one lint pass and thereby suppresses all other lints:
https://github.com/rust-lang/rust-clippy/blob/8baeb26754cd64baf7833b66ef5cf3f552911306/clippy_lints/src/lib.rs#L595-L600
Maybe such a mechanism could be used to actually measure the impact that a single lint has on Clippy's performance
Sorry for the delay, I totally forgot about this comment and was making a retrospect of the performance project when I stumbled on it again :sweat_smile:
Yeah, I executed a script similar to this:
for file in tests:
SPEEDTEST=ui cargo uitest | <text processing to get average time> > RESULTS$file
If there's a better, we have a server to get more accurate numbers!