mwparserfromhell icon indicating copy to clipboard operation
mwparserfromhell copied to clipboard

Incorrect SmartList slicing

Open nyurik opened this issue 6 years ago • 1 comments

The slice operation for the smartlist is incorrectly handled (inconsistent with the built-in python lists). This results in IDE crashes when I tried to examine internal variables, and could also cause people's code to behave incorrectly:

sample = ['abc']
for item in sample[0:2]:  # this works fine -- iterating one item
    print(item)

from mwparserfromhell import parse
code = parse('abc')
for item in code.nodes[0:2]: # key error - tries to access code.nodes[1] internally
    print(item)

It seems the internal slice is not clipped to the size of the smartlist, thus resulting in an error. I am not sure of the logic -- it does something magical in line 56 with the maxsize that doesn't seem right, but in the actual code path it takes (line 59) the clipping is not involved at all:

https://github.com/earwig/mwparserfromhell/blob/416c7f6b78dd9ea0c690a0a8f0cba764ccf0322b/mwparserfromhell/smart_list.py#L50-L60

The normalizer is called from here (the key param is start=0, end=2) https://github.com/earwig/mwparserfromhell/blob/416c7f6b78dd9ea0c690a0a8f0cba764ccf0322b/mwparserfromhell/smart_list.py#L94-L102

nyurik avatar Dec 31 '19 03:12 nyurik

I created a PR with several new unit tests to highlight related smart-list problems. The PR is obviously not passing because the PR does not actually change smart list code, only the unit tests.

How should we proceed? Please confirm these are the real bugs. Thx!

nyurik avatar Dec 31 '19 16:12 nyurik