edx-platform icon indicating copy to clipboard operation
edx-platform copied to clipboard

Fix flaky TestGetCourseUserPartitions tests

Open robrap opened this issue 1 year ago • 3 comments

Following the old Flaky Test Process, I've deleted two flaky tests.

The deletion PR is https://github.com/openedx/edx-platform/pull/35029.

The following tests were deleted:

  • TestGetCourseUserPartitions.test_enrollment_track_partition_not_added_if_conflict
    • Link to failure https://github.com/openedx/edx-platform/actions/runs/9616581879/job/26526428843 that passed after re-running.
  • TestGetCourseUserPartitions.test_enrollment_track_partition_not_added_if_disabled
    • Link to failure https://github.com/openedx/edx-platform/actions/runs/9641692755/job/26587890635 that passed after re-running.

Notes/Questions:

  • The only recent partition-related change seems to be: https://github.com/openedx/edx-platform/pull/33788
  • Separately, we might want to ticket updating the Flaky Test Process, but where?
    • I wrote up https://discuss.openedx.org/t/flaky-test-process/13263
  • The class has one last test, test_filter_inactive_user_partitions, which may also be flaky but I didn't see it yet so I left it in the code.

Failure 1:

=========================== short test summary info ============================
FAILED xmodule/partitions/tests/test_partitions.py::TestGetCourseUserPartitions::test_enrollment_track_partition_not_added_if_conflict - AssertionError: assert 1 == 2
 +  where 2 = len([UserPartition(id=0, name='Mock Partition', description='for testing purposes', groups=[Group(id=0, name='Group 1'), G...serPartitionScheme object at 0x7f3198774dd0>, parameters={'course_id': 'course-v1:org_0+course_0+run_0'}, active=True)])
==== 1 failed, 1487 passed, 170 skipped, 1572 warnings in 229.69s (0:03:49) ====

Failure 2:

=========================== short test summary info ============================
FAILED xmodule/partitions/tests/test_partitions.py::TestGetCourseUserPartitions::test_enrollment_track_partition_not_added_if_disabled - AssertionError: assert 1 == 2
 +  where 2 = len([UserPartition(id=0, name='Mock Partition', description='for testing purposes', groups=[Group(id=0, name='Group 1'), G...serPartitionScheme object at 0x7f6d54d9e9d0>, parameters={'course_id': 'course-v1:org_0+course_0+run_0'}, active=True)])
==== 1 failed, 1487 passed, 170 skipped, 1572 warnings in 229.98s (0:03:49) ====

robrap avatar Jun 24 '24 16:06 robrap

FYI: @mariajgrimaldi @kdmccormick

robrap avatar Jun 25 '24 15:06 robrap

Hi, @robrap! Thanks for letting us know. I'll try to review what happened this week.

mariajgrimaldi avatar Jun 25 '24 20:06 mariajgrimaldi

@mariajgrimaldi: I'm only guessing it is your PR because I didn't see any other changes, but I want to confirm it is just a guess.

robrap avatar Jun 25 '24 20:06 robrap

@mariajgrimaldi: Friendly reminder about this ticket. Thanks.

robrap avatar Aug 05 '24 13:08 robrap