mspyls: tons of unrelated completion candidates

[ianyepan]

This issue was originally over at https://github.com/emacs-lsp/lsp-python-ms/issues/79, but upon investigation, it seems that it could be a problem with company-lsp.

OS and emacs version:

I'm on macOS with Emacs 27.0.50

The problem only arises with mspyls in emacs. I've made sure by running pyls in emacs, as well as mspyls in VScode and none of the issues happened.

Description:

To reproduce the problem in a python buffer with mspyls + lsp-mode enabled, supposed we're trying to have autocompletion for print(). Start off by typing p followed by r and i, n etc. and we'll see the candidates narrow down and become fewer.

Specifically, at prin we'll see that only 5 candidates are present, namely, print, ImportWarning, StopIteration..... Of course, this is due to the fuzzy match result.

The strange thing now happens as soon as I hit t to complete print. Now with either mspyls in VSCode or pyls in emacs, only a single candidate print will remain in the tooltip. However, with mspyls in emacs, the tooltip is suddenly filled with hundreds (maybe more) of unrelated candidates such as if, else, bin, bool, break, AttributeError, compile, continue, copyright etc. As if it was a completion for a wildcard character.

Further investigation:

I have set company-lsp-cache-candidates to t, nil, and 'auto and compared the results. Here's my observation:

1. The behaviour I described in the title only happens if company-lsp-cache-candidates is set to 'auto.
2. If company-lsp-cache-candidates is set to t, I will miss several fuzzy-matched candidates, but no unrelated candidates show up either.
3. If set to nil, the completion works ideal as one would expect -- no unrelated candidates, and fuzzy-matched ones are present too. However, I feel that emacs slows down / lags a bit with this option.

[ianyepan]

As mentioned in https://github.com/emacs-lsp/lsp-python-ms/issues/79, @yyoncho provided a temporary workaround snippet to fix this issue. Example config:

(use-package company-lsp
  :commands company-lsp
  :config
  (setq company-lsp-cache-candidates 'auto)
  (add-to-list 'company-lsp-filter-candidates '(mspyls . t))
  (defun company-lsp--on-completion (response prefix)
    "Handle completion RESPONSE.
PREFIX is a string of the prefix when the completion is requested.
Return a list of strings as the completion candidates."
    (let* ((incomplete (and (hash-table-p response) (gethash "isIncomplete" response)))
           (items (cond ((hash-table-p response) (gethash "items" response))
                        ((sequencep response) response)))
           (candidates (mapcar (lambda (item)
                                 (company-lsp--make-candidate item prefix))
                               (lsp--sort-completions items)))
           (server-id (lsp--client-server-id (lsp--workspace-client lsp--cur-workspace)))
           (should-filter (or (eq company-lsp-cache-candidates 'auto) ; change from t to 'auto
                              (and (null company-lsp-cache-candidates)
                                   (company-lsp--get-config company-lsp-filter-candidates server-id)))))
      (when (null company-lsp--completion-cache)
        (add-hook 'company-completion-cancelled-hook #'company-lsp--cleanup-cache nil t)
        (add-hook 'company-completion-finished-hook #'company-lsp--cleanup-cache nil t))
      (when (eq company-lsp-cache-candidates 'auto)
        ;; Only cache candidates on auto mode. If it's t company caches the
        ;; candidates for us.
        (company-lsp--cache-put prefix (company-lsp--cache-item-new candidates incomplete)))
      (if should-filter
          (company-lsp--filter-candidates candidates prefix)
        candidates))))