Refactoring detection is sensitive to comments

[WesleyYue]

Environment data

• Language Server version: 2023.7.10
• OS and version: MacOS 13.2.1
• Python version (& distribution if applicable, e.g. Anaconda): 3.11.3, venv via Poetry

Code Snippet

def main():
    # here is a comment
    foo = "bar"

Repro Steps

1. Highlight the code from last character of line 3 to first character of line 2 (including the indentation spaces)
2. Ctrl + Shfit + R to trigger the refactoring menu

Expected behavior

Extract Method to be available

Actual behavior

"No refactorings available"

[IvanRancati]

Perhaps relevant: refactoring detection works correctly for ''' ''' style comments. The following will be refactored correctly, including the comment

def main():
    '''
    here is a comment
    ''' 
    foo = "bar"

My setup is slightly different from the one of the original bug report, but I can confirm the problem with # style comments

Pylance v2023.9.10 Python v2023.14.0 VS Code Version: 1.81.1 (system setup) OS: Windows_NT x64 10.0.22621

[debonte]

Looks like the core problem is that when verifyAndAdjustSelectionNodes looks for the start node of the selection range it doesn't take comments into account. Since the start of the comment isn't within a statement, the closest node that includes that point is the suite and that is rejected by the selectionContainsNode check.

I quickly hacked up a version of findNodeByOffset that took comments into account and called that from verifyAndAdjustSelectionNodes instead. That appeared to fix this issue, although the first line after the comment in the new method was indented by an extra level for some reason.

    static findNodeByOffsetWithComments(
        parseResults: ParseResults,
        node: ParseNode,
        offset: number
    ): ParseNode | undefined {
        let start = node.start;
        const indentationUtils = new IndentationUtils();
        const comment = indentationUtils.findCommentAtOrBeforeOffset(parseResults.tokenizerOutput.tokens, start);
        if (comment) {
            start = comment.start - 1;
        }
        if (offset < start || offset > TextRange.getEnd(node)) {
            return undefined;
        }
...

[debonte]

I quickly hacked up a version of findNodeByOffset that took comments into account and called that from verifyAndAdjustSelectionNodes instead.

@KacieKK, here's that hacky change: https://github.com/debonte/pyrx/commit/7435292c9429539521ed05a0d539714c958b1980