datatracker
datatracker copied to clipboard
ietf-tools
transient random name failure
type_defect | by [email protected]
FAIL: test_group_about_personnel (ietf.group.tests_info.GroupPagesTests.test_group_about_personnel)
Correct personnel should appear on the group About page
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/rjsparks/ietf/datatracker/rjs/7.27.1.dev0/ietf/group/tests_info.py", line 350, in test_group_about_personnel
self.assertContains(r, role.person.plain_name())
File "/Users/rjsparks/ietf/datatracker/rjs/7.27.1.dev0/env/lib/python3.7/site-packages/django/test/testcases.py", line 454, in assertContains
self.assertTrue(real_count != 0, msg_prefix + "Couldn't find %s in response" % text_repr)
AssertionError: False is not true : Couldn't find 'Mitchel O'Cadden' in response
Issue migrated from trac:3231 at 2022-03-04 08:14:05 +0000
@[email protected] commented
====================================================================== FAIL: test_api_upload_bluesheet (ietf.api.tests.CustomApiTests.test_api_upload_bluesheet)
Traceback (most recent call last): File "/home/rjsparks/ietf/datatracker/trunk/ietf/api/tests.py", line 245, in test_api_upload_bluesheet self.assertIn(html.escape(p['affiliation']), text) AssertionError: 'Mullin, O'Gowan and McGloin' not found in 'Bluesheet for IETF-82: acronym7 Sat-1100\n========================================================================\n\n6 attendees.\n\n\nAndrea Andreotti\tAzienda\nBosse Bernadotte\tBolaget AB\nCharles Charlemagne\tCompagnie\nMathilde Deschamps de Martinez\tRousset\nChristine Meara\tMullin, O'Gowan and McGloin\nPari Tara\tKar-Sengupta\n'
@[email protected] commented
escape failure
======================================================================
FAIL: test_api_upload_bluesheet (ietf.api.tests.CustomApiTests.test_api_upload_bluesheet)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/rjsparks/ietf/datatracker/rjs/7.35.1.dev0/ietf/api/tests.py", line 245, in test_api_upload_bluesheet
self.assertIn(html.escape(p['affiliation']), text)
AssertionError: 'שמעון-ח'טיב' not found in 'Bluesheet for IETF-82: acronym7 Thu-1100\n========================================================================\n\n6 attendees.\n\n\nAndrea Andreotti\tAzienda\nBosse Bernadotte\tBolaget AB\nCharles Charlemagne\tCompagnie\nעומר דמרי\tשמעון-ח'טיב\nΑιμιλία Κυριαννάκη\tΚρεμμύδας LLC\nІрина Затула\tШвачка, Бабиченко and Дробаха\n'
----------------------------------------------------------------------
====================================================================== FAIL: test_reject_reviewer_assignment (ietf.doc.tests_review.ReviewTests.test_reject_reviewer_assignment)
Traceback (most recent call last): File "/workspace/ietf/doc/tests_review.py", line 416, in test_reject_reviewer_assignment self.assertContains(r, assignment.reviewer.person.name) File "/home/dev/.local/lib/python3.9/site-packages/django/test/testcases.py", line 454, in assertContains self.assertTrue(real_count != 0, msg_prefix + "Couldn't find %s in response" % text_repr) AssertionError: False is not true : Couldn't find 'Tane O'Connor' in response
Author extraction failure. Author name used in test: नम्रता Author ascii name: Nmrtaa Author initials:
====================================================================== FAIL: test_submit_existing_txt_preserve_authors (ietf.submit.tests.SubmitTests.test_submit_existing_txt_preserve_authors)
Traceback (most recent call last): File "/workspace/ietf/submit/tests.py", line 722, in test_submit_existing_txt_preserve_authors self.submit_existing(["txt"], change_authors=False) File "/workspace/ietf/submit/tests.py", line 573, in submit_existing status_url, author = self.do_submission(name, rev, group, formats, author=prev_author.person) File "/workspace/ietf/submit/tests.py", line 218, in do_submission self.assertEqual(len(submission.authors), 1) AssertionError: 0 != 1
Is it possible to make the test runs repeatable, so these rare events can be more easily reproduced when they happen? I think you said in the past that the random seed for a run is being saved somewhere, but I don't know how one would start a new run with this seed, and whether that would actually cause the exact same sequence of events to happen.
I think that would be fantastic. It might be a bit nontrivial to track down the places we need to seed - I know we import random and use it in a number of factories as well as letting the faker library give us randomness. I'd think it's tractable, though.
Yes, we should add that infrastructure, BUT, for many of the above tests, the name that broke things is readily available, and the test can be run with just that name explicitly, so there's no need to wait for the ability to start with the same random state to make a lot of this better.
Temporarily change the place that builds the data for the test to only provide the specific input you want.
FAIL: test_group_about_personnel (ietf.group.tests_info.GroupPagesTests.test_group_about_personnel)
This one was fixed as part of 6ca6c35186bf42abf6b971691492f6d4be3428d1.
FAIL: test_api_upload_bluesheet (ietf.api.tests.CustomApiTests.test_api_upload_bluesheet)
This one was fixed as part of cf629a42ad53343734843a41e5db829d8aa1522b.
FAIL: test_reject_reviewer_assignment (ietf.doc.tests_review.ReviewTests.test_reject_reviewer_assignment)
This will be fixed by #4695.
#4695 claims to have fixed this, but I think
FAIL: test_submit_existing_txt_preserve_authors (ietf.submit.tests.SubmitTests.test_submit_existing_txt_preserve_authors)
may have a problem. Was that fixed yet?
Looking into it. One issue I found is #4798, but that is not causing that particular failure.
Another issue is that single-word names aren't handled correctly. I have a PR (#4799) pending for that.
We can start opening an issue per occurrence since these are becoming more rarified