pyahocorasick
pyahocorasick copied to clipboard
WojciechMula
Tests test_iter3 and test_iter2 fails on big endian
While building packages for openSUSE, test suite fails on all big endian architectures (ppc64, s390x, and armv7l):
[ 42s] ======================================================================
[ 42s] FAIL: test_iter3 (unittests.TestAutomatonIterSearch)
[ 42s] ----------------------------------------------------------------------
[ 42s] Traceback (most recent call last):
[ 42s] File "/home/abuild/rpmbuild/BUILD/pyahocorasick-1.4.1/unittests.py", line 775, i
n test_iter3
[ 42s] self.assertEqual(L, C)
[ 42s] AssertionError: Lists differ: [] != [(6, 'he'), (7, 'her'), (8, 'hers')]
[ 42s]
[ 42s] Second list contains 3 additional elements.
[ 42s] First extra element 0:
[ 42s] (6, 'he')
[ 42s]
[ 42s] - []
[ 42s] + [(6, 'he'), (7, 'her'), (8, 'hers')]
[ 42s]
[ 42s] ======================================================================
[ 42s] FAIL: test_iter2 (unittests.TestAutomatonIterSearchWithIgnoreWhiteSpace)
[ 42s] ----------------------------------------------------------------------
[ 42s] Traceback (most recent call last):
[ 42s] File "/home/abuild/rpmbuild/BUILD/pyahocorasick-1.4.1/unittests.py", line 890, i
n test_iter2
[ 42s] self.assertEqual(L, self.correct_positons_start_12)
[ 42s] AssertionError: Lists differ: [(4, 'she'), (4, 'he'), (6, 'her'), (8, 'he[49 chars
]he')] != [(13, 'he')]
[ 42s]
[ 42s] First differing element 0:
[ 42s] (4, 'she')
[ 42s] (13, 'he')
[ 42s]
[ 42s] First list contains 7 additional elements.
[ 42s] First extra element 1:
[ 42s] (4, 'he')
[ 42s]
[ 42s] - [(4, 'she'),
[ 42s] - (4, 'he'),
[ 42s] - (6, 'her'),
[ 42s] - (8, 'he'),
[ 42s] - (9, 'her'),
[ 42s] - (11, 'hers'),
[ 42s] - (13, 'she'),
[ 42s] - (13, 'he')]
[ 42s] ? ^
[ 42s]
[ 42s] + [(13, 'he')]
[ 42s] ? ^
Complete build on PPC64 with all details
@mcepl How can we debug this?
Tell me which logs and other information I should collect?
@mcepl re logs. Where are you building for openSUSE, on all big endian architectures (ppc64, s390x, and armv7l)? Could you help us by having a way to easily run the tests on demand or on commit on all these architectures? That would be awesome!
Last I checked in my cave, I could not find any leftover mainframe in a corner to use to run tests ;) :innocent:
@mcepl re logs. Where are you building for openSUSE, on all big endian architectures (ppc64, s390x, and armv7l)? Could you help us by having a way to easily run the tests on demand or on commit on all these architectures? That would be awesome!
See https://build.opensuse.org/package/show/devel:languages:python/python-pyahocorasick ... if I can just run the test suite, I can run it on all these (and probably some other) platforms.
@mcepl thanks... the basic test suite is with make test (for py2 and py3) or just make test3 for Python 3 only. that's for the latest master code and the latest 1.4.4 release. The set of extended tests are listed in https://github.com/WojciechMula/pyahocorasick/blob/master/release_checklist.txt
I see your run them with pytest per:
pytest-3.9 --ignore=_build.python39 --ignore=_build.python310 --ignore=_build.python38 -v -k '' unittests.py ... this works too. @WojciechMula FYI, I want to reorg the test suite to be also all under a test/ directory (a pet pieve of mine ;) )
I see your run them with pytest per:
pytest-3.9 --ignore=_build.python39 --ignore=_build.python310 --ignore=_build.python38 -v -k '' unittests.py ... this works too. @WojciechMula FYI, I want to reorg the test suite to be also all under a test/ directory (a pet pieve of mine ;) )
I have no particular problem with unittest v. pytest, but I am not big fan of other testing packages (nose2, or some other ones, and nose1 is dead and being removed from openSUSE this week).
Still having issue on s390x (Python 3.12, pyahocorasick-2.0.0)
=================================== FAILURES ===================================
______________________________ test_basic_unicode ______________________________
@pytest.mark.skipif(not ahocorasick.unicode, reason="Run only with unicode build")
@pytest.mark.xfail(on_windows, reason="Fails on Windows")
def test_basic_unicode():
automaton = ahocorasick.Automaton()
words = "he e hers his she hi him man he".split()
# 0 1 2 3 4 5 6 7 8
for i, w in enumerate(words):
automaton.add_word(w, (i, w))
query = "he rshershidamanza "
# 01234567890123
automaton.make_automaton()
assert query[2:8] == ' rsher'
results = list(automaton.iter(string=query, start=2, end=8))
> assert results == [(6, (4, 'she')), (6, (8, 'he')), (6, (1, 'e'))]
E AssertionError: assert [] == [(6, (4, 'she...(6, (1, 'e'))]
E Right contains 3 more items, first extra item: (6, (4, 'she'))
E Use -v to get more diff
tests/test_basic.py:115: AssertionError
_____________________________ test_issue_10_case_1 _____________________________
def test_issue_10_case_1():
ac = ahocorasick.Automaton()
ac.add_word(conv('S'), 1)
ac.make_automaton()
buffer = conv('SSS')
count = 0
for _item in ac.iter(buffer, 0, 3): # this causes an error
count += 1
> assert count == 3
E assert 0 == 3
tests/test_issue_10.py:23: AssertionError
_____________________________ test_issue_10_case_2 _____________________________
def test_issue_10_case_2():
ac = ahocorasick.Automaton()
ac.add_word(conv('S'), 1)
ac.make_automaton()
buffer = conv('SSS')
count = 0
for _item in ac.iter(buffer, 0, 2): # no error, but it misses the last 'S' in the buffer
count += 1
> assert count == 2
E assert 0 == 2
tests/test_issue_10.py:35: AssertionError
______________________ TestAutomatonIterSearch.test_iter3 ______________________
self = <test_unit.TestAutomatonIterSearch testMethod=test_iter3>
def test_iter3(self):
A = self.add_words_and_make_automaton()
start = 4
end = 9
C = []
for index, word in A.iter(conv(self.string[start:end])):
C.append((index + start, word))
L = []
for index, word in A.iter(conv(self.string), start, end):
L.append((index, word))
> self.assertEqual(L, C)
E AssertionError: Lists differ: [] != [(6, 'he'), (7, 'her'), (8, 'hers')]
E
E Second list contains 3 additional elements.
E First extra element 0:
E (6, 'he')
E
E - []
E + [(6, 'he'), (7, 'her'), (8, 'hers')]
tests/test_unit.py:737: AssertionError
____________ TestAutomatonIterSearchWithIgnoreWhiteSpace.test_iter2 ____________
self = <test_unit.TestAutomatonIterSearchWithIgnoreWhiteSpace testMethod=test_iter2>
def test_iter2(self):
self.add_words_and_make_automaton()
A = self.A
self.assertEqual(A.kind, ahocorasick.AHOCORASICK)
L = []
for index, word in A.iter(conv(self.string), ignore_white_space=True, start=12):
L.append((index, word))
> self.assertEqual(L, self.correct_positons_start_12)
E AssertionError: Lists differ: [(4, 'she'), (4, 'he'), (6, 'her'), (8, 'he[49 chars]he')] != [(13, 'he')]
E
E First differing element 0:
E (4, 'she')
E (13, 'he')
E
E First list contains 7 additional elements.
E First extra element 1:
E (4, 'he')
E
E - [(4, 'she'),
E - (4, 'he'),
E - (6, 'her'),
E - (8, 'he'),
E - (9, 'her'),
E - (11, 'hers'),
E - (13, 'she'),
E - (13, 'he')]
E ? ^
E
E + [(13, 'he')]
E ? ^
tests/test_unit.py:849: AssertionError
=========================== short test summary info ============================
FAILED tests/test_basic.py::test_basic_unicode - AssertionError: assert [] ==...
FAILED tests/test_issue_10.py::test_issue_10_case_1 - assert 0 == 3
FAILED tests/test_issue_10.py::test_issue_10_case_2 - assert 0 == 2
FAILED tests/test_unit.py::TestAutomatonIterSearch::test_iter3 - AssertionErr...
FAILED tests/test_unit.py::TestAutomatonIterSearchWithIgnoreWhiteSpace::test_iter2
=================== 5 failed, 158 passed, 7 skipped in 2.14s ===================
