AssertionError completing a trailer with a space after dot

[K-410]

>>> from jedi import __version__
>>> __version__
'0.19.0'

Also tested on 0.18.2.

Test code. See traceback at bottom:

src = "object. \n"

from jedi.api import Interpreter
Interpreter(src, []).complete(1, 8)

This seems to happen because in Completion._complete_python,

            elif nonterminals[-1] in ('trailer', 'dotted_name') and nodes[-1] == '.':
                dot = self._module_node.get_leaf_for_position(self._position)
                if dot.type == "endmarker":
                    # This is a bit of a weird edge case, maybe we can somehow
                    # generalize this.
                    dot = leaf.get_previous_leaf()
                cached_name, n = self._complete_trailer(dot.get_previous_leaf())

Specifically:

dot = self._module_node.get_leaf_for_position(self._position)

is gives us the newline leaf I'm assuming Jedi expects dot to be "." operator. But we added a space, which is still valid python. And there's a newline, but jedi only accounts for endmarker.

>>> eval("object.      mro")
>>> <built-in method mro of type object at 0x00007FFFA9EEBBB0>

When self._complete_trailer is called with dot.get_previous_leaf(), what's actually being passed is the dot operator.

cached_name, n = self._complete_trailer(dot.get_previous_leaf())  # dot.get_previous_leaf() == the dot operator.

This results in the following branch in syntax_tree._infer_node() to be taken, throwing AssertionError.

    elif typ == 'operator':
        # Must be an ellipsis, other operators are not inferred.
        if element.value != '...':
            origin = element.parent
            raise AssertionError("unhandled operator %s in %s " % (repr(element.value), origin))
        return ValueSet([compiled.builtin_from_name(inference_state, 'Ellipsis')])

The traceback

Traceback (most recent call last):
  File "Z:\py_jedi\main.py", line 7, in <module>
    Interpreter(src, []).complete(1, 8)
  File "z:\py_jedi\Lib\site-packages\jedi\api\helpers.py", line 487, in wrapper  
    return func(self, line, column, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\api\__init__.py", line 214, in complete
    return completion.complete()
           ^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\api\completion.py", line 170, in complete
    cached_name, completion_names = self._complete_python(leaf)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\api\completion.py", line 284, in _complete_python
    cached_name, n = self._complete_trailer(dot.get_previous_leaf())
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\api\completion.py", line 385, in _complete_trailer
    values = infer_call_of_leaf(inferred_context, previous_leaf)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\inference\helpers.py", line 79, in infer_call_of_leaf
    return context.infer_node(leaf)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\inference\context.py", line 224, in infer_node
    return infer_node(self, node)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\inference\syntax_tree.py", line 157, in infer_node
    return _infer_node_if_inferred(context, element)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\inference\syntax_tree.py", line 170, in _infer_node_if_inferred
    return _infer_node_cached(context, element)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\inference\cache.py", line 44, in wrapper
    rv = function(obj, *args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\inference\syntax_tree.py", line 175, in _infer_node_cached
    return _infer_node(context, element)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\debug.py", line 81, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\inference\syntax_tree.py", line 83, in wrapper
    return func(context, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "z:\py_jedi\Lib\site-packages\jedi\inference\syntax_tree.py", line 230, in _infer_node
    raise AssertionError("unhandled operator %s in %s " % (repr(element.value), origin))
AssertionError: unhandled operator '.' in <PythonErrorNode: object.@1,0>

Possible fix could be to check if dot.type is either "endmarker" or "newline" before trying the previous leaf. I don't have tests setup.

                dot = self._module_node.get_leaf_for_position(self._position)
                if dot.type in {"endmarker", "newline"}:
                    dot = leaf.get_previous_leaf()

[davidhalter]

I suspect that the reason for this is that parso's error recovery is a bit weird and unaccounted for. Would have to check this in detail.