datatracker icon indicating copy to clipboard operation
datatracker copied to clipboard

Transient test failures

Open rjsparks opened this issue 1 year ago • 16 comments

We have some transient test failures creeping back in. This issue will gather them.

rjsparks avatar Jun 15 '23 16:06 rjsparks

======================================================================

FAIL: test_edit (ietf.secr.sreq.tests.SessionRequestTestCase.test_edit)

Traceback (most recent call last): File "/__w/datatracker/datatracker/ietf/secr/sreq/tests.py", line 160, in test_edit self.assertFalse(sessions[0].joint_with_groups.count()) AssertionError: 2 is not false


rjsparks avatar Jun 15 '23 16:06 rjsparks

I just hit the above as well.

larseggert avatar Jun 21 '23 08:06 larseggert

====================================================================== FAIL: test_review_requests (ietf.group.tests_review.ReviewTests.test_review_requests)

Traceback (most recent call last): File "/workspace/ietf/group/tests_review.py", line 44, in test_review_requests self.assertContains(r, assignment.reviewer.person.name) File "/home/dev/.local/lib/python3.9/site-packages/django/test/testcases.py", line 660, in assertContains self.assertTrue( AssertionError: False is not true : Couldn't find 'Bjorn van 't Wel' in response

rjsparks avatar Jul 08 '23 18:07 rjsparks

@rjsparks the ietf.group.tests_review.ReviewTests.test_review_requests failure should be fixed by #5986. The issue was that the ' got HTML-escaped to '.

larseggert avatar Jul 18 '23 13:07 larseggert

@larseggert - Makes me wonder - Do we have an open path already (and does this make it any worse) for arbitrary HTML injection through the user provided name? Maybe this needs bleaching before being marked clean on the way out?

rjsparks avatar Jul 18 '23 13:07 rjsparks

Possibly. The other way to fix this is of course to not mark it safe and instead compare against the escaped name in the test. (We would need to temporarily hardwire the factory to that name to find all the tests that are broken in the same way.)

larseggert avatar Jul 18 '23 13:07 larseggert

@rjsparks Which fix do you prefer?

larseggert avatar Jul 18 '23 17:07 larseggert

Better not to mark it safe unless we actually mean to allow literal HTML in the fields - I'm fairly sure we want Django to do the escaping here and we should update the tests to expect that.

jennifer-richards avatar Jul 19 '23 18:07 jennifer-richards

Ok. Will update the PR at the code sprint

larseggert avatar Jul 19 '23 18:07 larseggert

Done

larseggert avatar Jul 21 '23 15:07 larseggert

Here is another:

======================================================================
FAIL: test_edit (ietf.secr.sreq.tests.SessionRequestTestCase.test_edit)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/__w/datatracker/datatracker/ietf/secr/sreq/tests.py", line 168, in test_edit
    self.assertContains(r, 'Second session with: {} {}'.format(group3.acronym, group4.acronym))
  File "/usr/local/lib/python3.9/site-packages/django/test/testcases.py", line 660, in assertContains
    self.assertTrue(
AssertionError: False is not true : Couldn't find 'Second session with: acronym5258 acronym5260' in response

larseggert avatar Aug 10 '23 11:08 larseggert

Same error as above in this run: https://github.com/ietf-tools/datatracker/actions/runs/5902988978/job/16012002249

======================================================================
FAIL: test_edit (ietf.secr.sreq.tests.SessionRequestTestCase.test_edit)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/__w/datatracker/datatracker/ietf/secr/sreq/tests.py", line 168, in test_edit
    self.assertContains(r, 'Second session with: {} {}'.format(group3.acronym, group4.acronym))
  File "/usr/local/lib/python3.9/site-packages/django/test/testcases.py", line 660, in assertContains
    self.assertTrue(
AssertionError: False is not true : Couldn't find 'Second session with: acronym5270 acronym5272' in response

larseggert avatar Aug 18 '23 13:08 larseggert

ietf.secr.sreq.tests.SessionRequestTestCase.test_edit was fixed in #6394

larseggert avatar Oct 03 '23 09:10 larseggert

FAIL: test_upcoming_view_filter_whitespace (ietf.meeting.tests_js.InterimTests) 
Whitespace in filter lists should be ignored 
---------------------------------------------------------------------- 
Traceback (most recent call last): 
 File "/__w/datatracker/datatracker/ietf/meeting/tests_js.py", line 1195, in test_upcoming_view_filter_whitespace 
 self.do_upcoming_view_filter_test('?show=mars , ames &hide= ames', self.displayed_interims(groups=['mars'])) 
 File "/__w/datatracker/datatracker/ietf/meeting/tests_js.py", line 1048, in do_upcoming_view_filter_test 
 self.assert_upcoming_meeting_visibility(visible_meetings) 
 File "/__w/datatracker/datatracker/ietf/meeting/tests_js.py", line 1003, in assert_upcoming_meeting_visibility 
 self.assertEqual(unexpected, set(), "Unexpected row visible") 
AssertionError: Items in the first set but not the second: 
'interim-2023-sg-01' : Unexpected row visible

(also randomly affects other tests using the same helper method)

jennifer-richards avatar Nov 29 '23 20:11 jennifer-richards

 ======================================================================
FAIL: test_complete_review_upload_content (ietf.doc.tests_review.ReviewTests.test_complete_review_upload_content)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/__w/datatracker/datatracker/ietf/doc/tests_review.py", line 838, in test_complete_review_upload_content
    self.assertIn(assignment.reviewer.role_set.filter(group=assignment.review_request.team,name='reviewer').first().person.plain_name(), parseaddr(outbox[0]["From"])[0] )
AssertionError: 'எழில்மன்னன் தங்கராஐ;' not found in 'எழில்மன்னன் via Datatracker'

rjsparks avatar Mar 04 '24 22:03 rjsparks

======================================================================
FAIL: test_session_attendance (ietf.meeting.tests_views.ProceedingsTests.test_session_attendance)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/__w/datatracker/datatracker/ietf/meeting/tests_views.py", line 8601, in test_session_attendance
    self.assertContains(r, person.name)
  File "/usr/local/lib/python3.9/site-packages/django/test/testcases.py", line 660, in assertContains
    self.assertTrue(
AssertionError: False is not true : Couldn't find 'Arthur da Mata' in response

This is resolved by #7198

rjsparks avatar Mar 16 '24 03:03 rjsparks