rav1e icon indicating copy to clipboard operation
rav1e copied to clipboard

0.5.1: test failures on ppc64el

Open sebastinas opened this issue 3 years ago • 3 comments
trafficstars

We have seen test failures on Debian's ppc64el architecture. The failing test are all test_encode_decode.

Detailed output:

running 163 tests test api::test::flush_low_latency_scene_change_detection ... ok test api::test::flush_low_latency_no_scene_change ... ok test api::test::flush_unlimited_low_latency_no_scene_change ... ok test api::test::flush_unlimited_low_latency_scene_change_detection ... ok test api::test::flush_reorder_scene_change_detection ... ok test api::test::guess_frame_subtypes_assert ... ok test api::test::large_width_assert ... ok test api::test::log_q_exp_overflow ... ok test api::test::flush_reorder_no_scene_change ... ok test api::test::flush_unlimited_reorder_scene_change_detection ... ok test api::test::flush_unlimited_reorder_no_scene_change ... ok test api::test::lookahead_size_properly_bounded_10 has been running for over 60 seconds test api::test::lookahead_size_properly_bounded_16 has been running for over 60 seconds test api::test::lookahead_size_properly_bounded_lowlatency_1 ... ok test api::test::lookahead_size_properly_bounded_10 ... ok test api::test::max_key_frame_interval_overflow ... ok test api::test::lookahead_size_properly_bounded_16 ... ok test api::test::lookahead_size_properly_bounded_lowlatency_8 ... ok test api::test::minimum_frame_delay ... ok test api::test::output_frameno_incremental_reorder_keyframe_at_0 ... ok test api::test::output_frameno_incremental_reorder_keyframe_at_1 ... ok test api::test::output_frameno_incremental_reorder_keyframe_at_2 ... ok test api::test::output_frameno_incremental_reorder_keyframe_at_3 ... ok test api::test::output_frameno_incremental_reorder_keyframe_at_4 ... ok test api::test::output_frameno_incremental_reorder_minus_0 ... ok test api::test::output_frameno_incremental_reorder_minus_1 ... ok test api::test::output_frameno_incremental_reorder_minus_2 ... ok test api::test::output_frameno_incremental_reorder_minus_3 ... ok test api::test::output_frameno_incremental_reorder_minus_4 ... ok test api::test::output_frameno_incremental_reorder_scene_change_at_0 ... ok test api::test::lookahead_size_properly_bounded_8 has been running for over 60 seconds test api::test::output_frameno_incremental_reorder_scene_change_at_1 ... ok test api::test::output_frameno_incremental_reorder_scene_change_at_2 ... ok test api::test::output_frameno_incremental_reorder_scene_change_at_3 ... ok test api::test::output_frameno_incremental_reorder_scene_change_at_4 ... ok test api::test::output_frameno_low_latency_minus_0 ... ok test api::test::output_frameno_low_latency_minus_1 ... ok test api::test::output_frameno_no_scene_change_at_flash_smaller_than_max_len_flash ... ok test api::test::output_frameno_no_scene_change_at_short_flash_1 ... ok test api::test::output_frameno_no_scene_change_at_short_flash_2 ... ok test api::test::output_frameno_no_scene_change_at_short_flash_3 ... ok test api::test::output_frameno_reorder_minus_0 ... ok test api::test::output_frameno_reorder_minus_1 ... ok test api::test::output_frameno_reorder_minus_2 ... ok test api::test::max_quantizer_bounds_correctly ... ok test api::test::output_frameno_reorder_minus_4 ... ok test api::test::output_frameno_reorder_minus_3 ... ok test api::test::output_frameno_reorder_scene_change_at_1 ... ok test api::test::output_frameno_reorder_scene_change_at_0 ... ok test api::test::output_frameno_reorder_scene_change_at_2 ... ok test api::test::output_frameno_reorder_scene_change_at_3 ... ok test api::test::min_quantizer_bounds_correctly ... ok test api::test::output_frameno_reorder_scene_change_at_4 ... ok test api::test::pyramid_level_low_latency_minus_0 ... ok test api::test::pyramid_level_low_latency_minus_1 ... ok test api::test::output_frameno_scene_change_after_multiple_flashes ... ok test api::test::pyramid_level_reorder_minus_0 ... ok test api::test::pyramid_level_reorder_minus_1 ... ok test api::test::output_frameno_scene_change_before_flash_longer_than_max_flash_len ... ok test api::test::pyramid_level_reorder_minus_4 ... ok test api::test::pyramid_level_reorder_minus_3 ... ok test api::test::pyramid_level_reorder_minus_2 ... ok test api::test::lookahead_size_properly_bounded_8 ... ok test api::test::pyramid_level_reorder_scene_change_at_1 ... ok test api::test::pyramid_level_reorder_scene_change_at_0 ... ok test api::test::rdo_lookahead_frames_overflow ... ok test api::test::reservoir_max_overflow ... ok test api::test::pyramid_level_reorder_scene_change_at_2 ... ok test api::test::target_bitrate_overflow ... ok test api::test::pyramid_level_reorder_scene_change_at_3 ... ok test api::test::tile_cols_overflow ... ok test api::test::time_base_den_divide_by_zero ... ok test api::test::zero_frames ... ok test api::test::zero_width ... ok test cdef::rust::test::check_max_element ... ok test context::partition_unit::test::cdf_map ... ok test context::partition_unit::test::cfl_joint_sign ... ok test api::test::switch_frame_interval ... ok test dist::test::get_sad_same_u16 ... ok test api::test::pyramid_level_reorder_scene_change_at_4 ... ok test dist::test::get_sad_same_u8 ... ok test ec::test::booleans ... ok test ec::test::cdf ... ok test ec::test::mixed ... ok test encoder::test::check_partition_types_order ... ok test predict::test::pred_matches_u8 ... ok test predict::test::pred_max ... ok test quantize::test::gen_divu_table ... ok test dist::test::get_satd_same_u16 ... ok test quantize::test::test_tx_log_scale ... ok test rate::test::bexp64_vectors ... ok test quantize::test::test_divu_pair ... ok test rate::test::blog64_vectors ... ok test rdo::estimate_rate_test ... ok test test_encode_decode::bitrate_aom ... ignored test test_encode_decode::chroma_sampling_400_aom ... ignored test test_encode_decode::chroma_sampling_420_aom ... ignored test test_encode_decode::chroma_sampling_422_aom ... ignored test test_encode_decode::chroma_sampling_444_aom ... ignored test dist::test::get_satd_same_u8 ... ok test api::test::test_opaque_delivery ... ok test test_encode_decode::high_bit_depth_10_aom ... ignored test test_encode_decode::high_bit_depth_12_aom ... ignored test rate::test::blog64_bexp64_round_trip ... ok test test_encode_decode::error_resilient_aom ... FAILED test test_encode_decode::error_resilient_reordering_aom ... FAILED test test_encode_decode::keyframes_aom ... FAILED test test_encode_decode::low_bit_depth_aom ... ignored test test_encode_decode::odd_size_frame_with_full_rdo_aom ... ignored test test_encode_decode::quantizer_100_aom ... FAILED test test_encode_decode::quantizer_120_aom ... FAILED test test_encode_decode::quantizer_60_aom ... FAILED test test_encode_decode::quantizer_80_aom ... FAILED test test_encode_decode::rdo_loop_decision_cdef_sanity_aom ... ignored test test_encode_decode::rdo_loop_decision_lrf_sanity_aom ... ignored test test_encode_decode::reordering_aom ... FAILED test test_encode_decode::reordering_short_video_aom ... FAILED test test_encode_decode::large_dimension::dimension_1024x1024_aom has been running for over 60 seconds test test_encode_decode::large_dimension::dimension_2048x2048_aom has been running for over 60 seconds test test_encode_decode::small_dimension::dimension_256x256_aom ... FAILED test test_encode_decode::large_dimension::dimension_512x512_aom has been running for over 60 seconds test test_encode_decode::large_dimension::dimension_512x512_aom ... FAILED test test_encode_decode::small_dimension::dimension_258x258_aom ... FAILED test test_encode_decode::small_dimension::dimension_260x260_aom ... FAILED test test_encode_decode::small_dimension::dimension_262x262_aom ... FAILED test test_encode_decode::small_dimension::dimension_264x264_aom ... FAILED test test_encode_decode::speed_0_aom ... ignored test test_encode_decode::speed_10_aom ... ignored test test_encode_decode::speed_1_aom ... ignored test test_encode_decode::speed_2_aom ... ignored test test_encode_decode::speed_3_aom ... ignored test test_encode_decode::speed_4_aom ... ignored test test_encode_decode::speed_5_aom ... ignored test test_encode_decode::speed_6_aom ... ignored test test_encode_decode::speed_7_aom ... ignored test test_encode_decode::speed_8_aom ... ignored test test_encode_decode::speed_9_aom ... ignored test test_encode_decode::small_dimension::dimension_265x265_aom ... FAILED test test_encode_decode::switch_frame_aom ... FAILED test test_encode_decode::tile_encoding_with_stretched_restoration_units_aom ... FAILED test test_encode_decode::tiny_dimension::dimension_128x128_aom ... FAILED test test_encode_decode::tiny_dimension::dimension_16x16_aom ... ok test test_encode_decode::tiny_dimension::dimension_1x1_aom ... ok test test_encode_decode::tiny_dimension::dimension_2x2_aom ... ok test test_encode_decode::tiny_dimension::dimension_32x32_aom ... ok test test_encode_decode::tiny_dimension::dimension_4x4_aom ... ok test test_encode_decode::tiny_dimension::dimension_64x64_aom ... FAILED test test_encode_decode::tiny_dimension::dimension_8x8_aom ... ok test tiling::plane_region::area_test ... ok test tiling::plane_region::frame_block_offset ... ok test tiling::tiler::test::from_target_tiles_422 ... ok test tiling::tiler::test::test_tile_area ... ok test tiling::tiler::test::test_tile_blocks_area ... ok test tiling::tiler::test::test_tile_blocks_write ... ok test tiling::tiler::test::test_tile_iter_len ... ok test tiling::tiler::test::test_tile_motion_vectors_write ... ok test tiling::tiler::test::test_tile_restoration_edges ... ok test tiling::tiler::test::test_tile_restoration_write ... ok test tiling::tiler::test::test_tile_write ... ok test tiling::tiler::test::test_tiling_info_from_tile_count ... ok test tiling::tiler::test::tile_log2_overflow ... ok test transform::test::log_tx_ratios ... ok test transform::test::roundtrips_u16 ... ok test transform::test::roundtrips_u8 ... ok test util::align::test::sanity_heap ... ok test util::align::test::sanity_stack ... ok test test_encode_decode::still_picture_mode_aom has been running for over 60 seconds test test_encode_decode::large_dimension::dimension_1024x1024_aom ... FAILED test test_encode_decode::still_picture_mode_aom ... FAILED test test_encode_decode::large_dimension::dimension_2048x2048_aom ... FAILED

failures:

---- test_encode_decode::error_resilient_aom stdout ---- thread 'test_encode_decode::error_resilient_aom' panicked at 'assertion failed: (left == right): at row 16 of plane 2

[1mDiff[0m [31m< left[0m / [32mright >[0m : [ [31m< 81,[0m [31m< 254,[0m [31m< 4,[0m [31m< 186,[0m [31m< 219,[0m [31m< 133,[0m [31m< 58,[0m [31m< 234,[0m [31m< 230,[0m [31m< 90,[0m [31m< 89,[0m [31m< 237,[0m [31m< 139,[0m [31m< 83,[0m [31m< 73,[0m [31m< 43,[0m [31m< 179,[0m [31m< 103,[0m [31m< 80,[0m [31m< 121,[0m [31m< 24,[0m [31m< 156,[0m [31m< 66,[0m [31m< 121,[0m [32m> 67,[0m [32m> 241,[0m [32m> 0,[0m [32m> 170,[0m 201, [31m< 100,[0m [31m< 5,[0m [31m< 9,[0m [31m< 157,[0m [31m< 142,[0m [31m< 17,[0m [31m< 76,[0m [32m> 120,[0m [32m> 42,[0m [32m> 212,[0m [32m> 216,[0m [32m> 77,[0m [32m> 71,[0m [32m> 220,[0m [32m> 119,[0m [32m> 74,[0m [32m> 57,[0m [32m> 27,[0m [32m> 170,[0m [32m> 94,[0m [32m> 71,[0m [32m> 112,[0m [32m> 15,[0m [32m> 147,[0m [32m> 57,[0m [32m> 112,[0m [32m> 192,[0m [32m> 91,[0m [32m> 0,[0m [32m> 0,[0m [32m> 148,[0m [32m> 133,[0m [32m> 8,[0m [32m> 67,[0m ]

', src/test_encode_decode/mod.rs:160:5 stack backtrace: 0: rust_begin_unwind at /usr/src/rustc-1.60.0/library/std/src/panicking.rs:584:5 1: core::panicking::panic_fmt at /usr/src/rustc-1.60.0/library/core/src/panicking.rs:143:14 2: rav1e::test_encode_decode::compare_plane at ./src/test_encode_decode/mod.rs:160:5 3: rav1e::test_encode_decode::aom::compare_img at ./src/test_encode_decode/aom.rs:168:7 4: <rav1e::test_encode_decode::aom::AomDecoder<T> as rav1e::test_encode_decode::TestDecoder<T>>::decode_packet at ./src/test_encode_decode/aom.rs:108:11 5: rav1e::test_encode_decode::TestDecoder::encode_decode at ./src/test_encode_decode/mod.rs:124:17 6: rav1e::test_encode_decode::error_resilient at ./src/test_encode_decode/mod.rs:511:3 7: rav1e::test_encode_decode::error_resilient_aom at ./src/test_encode_decode/mod.rs:500:37 8: rav1e::test_encode_decode::error_resilient_aom::{{closure}} at ./src/test_encode_decode/mod.rs:500:37 9: core::ops::function::FnOnce::call_once at /usr/src/rustc-1.60.0/library/core/src/ops/function.rs:227:5 10: core::ops::function::FnOnce::call_once at /usr/src/rustc-1.60.0/library/core/src/ops/function.rs:227:5 note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace.

// and many more failures like the above

failures: test_encode_decode::error_resilient_aom test_encode_decode::error_resilient_reordering_aom test_encode_decode::keyframes_aom test_encode_decode::large_dimension::dimension_1024x1024_aom test_encode_decode::large_dimension::dimension_2048x2048_aom test_encode_decode::large_dimension::dimension_512x512_aom test_encode_decode::quantizer_100_aom test_encode_decode::quantizer_120_aom test_encode_decode::quantizer_60_aom test_encode_decode::quantizer_80_aom test_encode_decode::reordering_aom test_encode_decode::reordering_short_video_aom test_encode_decode::small_dimension::dimension_256x256_aom test_encode_decode::small_dimension::dimension_258x258_aom test_encode_decode::small_dimension::dimension_260x260_aom test_encode_decode::small_dimension::dimension_262x262_aom test_encode_decode::small_dimension::dimension_264x264_aom test_encode_decode::small_dimension::dimension_265x265_aom test_encode_decode::still_picture_mode_aom test_encode_decode::switch_frame_aom test_encode_decode::tile_encoding_with_stretched_restoration_units_aom test_encode_decode::tiny_dimension::dimension_128x128_aom test_encode_decode::tiny_dimension::dimension_64x64_aom

test result: FAILED. 118 passed; 23 failed; 22 ignored; 0 measured; 0 filtered out; finished in 1084.85s

The full log is available at https://ci.debian.net/data/autopkgtest/testing/ppc64el/r/rust-rav1e/26190748/log.gz.

sebastinas avatar Sep 17 '22 12:09 sebastinas

I note that the _dav1d variants of the listed test failures also failed, recorded in the detailed log. This makes it unlikely to be a decoder bug and more likely a desync in rav1e on this platform.

barrbrain avatar Sep 17 '22 12:09 barrbrain

I noticed in some of the debian patches that features were removed from the crate, mostly for features that have not been stabilised. However, we also have some features for internal development: desync_finder, dump_ivf and dump_lookahead_data. These are used for investigating test failures and psychovisual issues. They are not intended for general use. At least some of these test failures are related to one of these features being enabled.

barrbrain avatar Sep 17 '22 13:09 barrbrain

Good to know. I'll remove them as well. I had already marked these failures to be ignored. The test infrastructure just simply tries to run the tests with all the features it finds.

sebastinas avatar Sep 17 '22 13:09 sebastinas