serpextract icon indicating copy to clipboard operation
serpextract copied to clipboard

fix flakt test of test_serps.py::TestSERPs::test_mojeek

Open lonly7star opened this issue 2 years ago • 0 comments

This PR aims to fix the flaky test on test_serps.py::TestSERPs::test_mojeek so the test could pass single run, multiple test run, and random test run. I'm doing this for a course project as a practice.

The result

The test won't pass on the single run with pytest, the following error raised > self.assertValidSERPs(serps, use_naive_method=True) > self.assertEqual(res.keyword, expected_keyword) E AttributeError: 'NoneType' object has no attribute 'keyword'

Steps to reproduce the issue

  1. run the test file with pytest test_serps.py

Issue of the code

The test tried to extract keyword from mojeek.com,("https://mojeek.com", "Mojeek", ""), which gives a None from the extraction. However, the assertValidSERP function tries to use this assertion self.assertEqual(res.keyword, expected_keyword) but since the result res is None so the assertion failed by trying to access the keyword element of a None object.

Proposed solution

Although the original test has the comment this test passes because of a manual change to the generated search engines JSON file. the next time that file is regenerated, this test will break. , I do have a proposed solution to include the None return keyword situation in the test. I adding an if statement if res is None: to check if the result is None before trying to access the element of it. if the result is None, then we compare the expect keyword to see if it is empty string self.assertEqual('', expected_keyword) and assert false for the url serp self.assertFalse(is_serp(url, **kwargs)). Using the else to include original assert statement. In this case, we could also compare the empty keyword return. I had run the whole test suit and the rest of test still passed. I also test it with pytest --flake-finder which is a re-run tool also detect flaky test in multiple runs. It still passed the multi-run detection. You can install it via pip install pytest-flakefinder if interested.

lonly7star avatar Nov 01 '21 23:11 lonly7star