Unexpected token

[ukos-git]

I get a lot of errors like Unexpected token of type "OP" encountered at valid (but uncommonly strange-looking) python code. The code is formatted PEP compliant.

        map(
            lambda (x, y): '{:s} = {:.3f}um'.format(
                x, y * 1e6
            ), dict(
                {axis: position[axis] for axis in axes}
            ).iteritems()
        )

I suspect this is an issue with the tokenizer used by SourceTrail that somehow misses to detect the opening and closing brackets correctly.

I hope the above example suffices as a mwe.

[mlangkabel]

This looks like a parser issue. I'm not an expert, but are you really supposed to use ( and ) after the lambda keyword?

[ukos-git]

I missed to mention that this is python2 code.

I would expect that unparenthesized tuples make it harder for your tokenizer to recognize operator precedence.

I am also not sure if varargslist to lambda would actually conflict with the function parameter argslist of map. So that the function parameters actually have precedence. However, lambdadef also waits for a colon after varargslist, so I guess the brackets in fpdef are always optional.

https://docs.python.org/2/reference/grammar.html parameters: '(' [varargslist] ')' varargslist: ((fpdef ['=' test] ',')* ('' NAME [',' '' NAME] | '' NAME) | fpdef ['=' test] (',' fpdef ['=' test]) [',']) fpdef: NAME | '(' fplist ')' fplist: fpdef (',' fpdef)* [','] ... lambdef: 'lambda' [varargslist] ':' test

Nevertheless, as I mentioned before, the above python code is valid.