Composite backend ((company-capf company-files)) does not work.

[ghost]

How to reproduce: File

int main( int argc, char** argv )
{
  std::string a = "./001_";
  return 0;
}

Create any file starting from 001_ . In my case it was 001_test.txt

company-diag:

Emacs 27.2 (x86_64-slackware-linux-gnu) of 2021-10-18 on laptop.lockywolf.net
Company 0.9.13

company-backends: ((company-capf company-files))

Used backend: (company-capf company-files)

Value of c-a-p-f: (eglot-completion-at-point t)
Major mode: c++-mode
Prefix: ("001_" . t)

Completions: none

Describe the issue

M-x company-manual-begin RET does not produce candidates. However, company-files does produce candidates. Setting company-backends to (company-files) produces candidates as well.

I tried edebug on company's code, and found that the two backends produce diffent prefixes. company-files produces ./001_, company-capf produces ("001_" . t). Not sure if it is related.

Expected behavior

See company-files' list of candidates. (As the c-a-p-f's one would be empty).

The error backtrace

No Emacs error present.

Additional context

I use eglot for IDE-like functionality, and only using company-capf works just fine, produces the list of candidates as expected. It seems to me that it is the merging functionality that is somehow self-contradictory.

[mashomee]

+1

[yugaego]

I tried edebug on company's code, and found that the two backends produce diffent prefixes. company-files produces ./001_, company-capf produces ("001_" . t). Not sure if it is related.

I don't have a fully wrapped response, but the prefixes are important in regard to getting results from the grouped backends. See, for instance: https://github.com/company-mode/company-mode/discussions/1149.

[dgutov]

@yugaego's link is on point: it's unfortunately a known problem. But given the specific example, it's a wonder people haven't been complaining about it more.

Nevertheless, to try to side-step the problem, @lockywolf have you tried not putting them in the same group? Like putting company-files after company-capf. Or before, if the said capf claims completion inside strings as well. Example:

(setq company-backends '(company-capf company-files)

[ghost]

This wouldn't make the suggestions appear in the same list, would it?

I didn't deliberately test it, but it seems to work, at least in the bash-mode (not in eglot-mode). Bash suggestions and files-suggestions "seem" to work.

[dgutov]

This wouldn't make the suggestions appear in the same list, would it?

It would not. Which would be a problem if both -capf and -files completions were appropriate at the same time in some particular context.

[yugaego]

Which would be a problem if both -capf and -files completions were appropriate at the same time in some particular context.

I agree with this statement. In practice (so far), I found it convenient to initiate company-files on purpose by calling company-other-backend (by hitting a custom keys binding), or specifically with M-x company-files.

The same time, wasn't :with keyword designed to solve this problem: show completions from "primary" (context-aware) and "secondary" (context-agnostic) backends? Such that '((company-capf :with company-files)) may be a solution then?

[dgutov]

The same time, wasn't :with keyword designed to solve this problem: show completions from "primary" (context-aware) and "secondary" (context-agnostic) backends? Such that '((company-capf :with company-files)) may be a solution then?

The goal was orthogonal: to ensure the fall-through to the next backend (or group) when none of the "primary" backends return a non-nil prefix.

Our inability to deal with non-equal prefixes in grouped backends (with one longer than another, like in this example) is more of a technical issue.

[yugaego]

Our inability to deal with non-equal prefixes in grouped backends (with one longer than another, like in this example) is more of a technical issue.

Then, can this issue be marked as a duplicate of the wishlist issue #426?

[dgutov]

Yes, indeed. Let's keep it open, though.

What would be the ideal behavior here? I don't know if having completions from company-capf and company-files at the same time is realistic, but maybe (company-capf company-yasnippet) is a better example. As described in #1268, yasnippet can even give different "prefixes" per completion, but suppose we just used the longest one.

How would the popup look if company-capf says the prefix is e and company-yasnippet says the prefix is <e? Do we pad the completions coming from company-capf with spaces? Do we prepend them with <? Do we not bother with lining up vertically at all, and just store the necessary adjustment (when inserting) internally?

Options 1 and 3 might lead to some breakage in the current implementation where we usually expect that every character from the prefix is present in all completions strings somewhere. Though that might require fixing only a few of such places.

Or we introduce full-on support for per-completion prefixes. The question about how the popup should look would remain, though.

[ryanobjc]

Ran in to some variant of this bug. Company-capf with lsp really messes things up basically. In the terraform lsp it tends to make the prefix smaller, often "", so company-files is totally useless even grouped because of the mismatched prefix thing as above. I have a case where company-capf produces no completions, sets the prefix to "" and company didn't seem to even try the next backend. I guess the backend "handled" things by offering no suggestions?

My company-diag says:

Emacs 29.0.91 (aarch64-apple-darwin21.6.0) of 2023-05-28 on m2.local
Company 0.9.13

company-backends: ((company-capf :with company-dabbrev-code company-files)
 company-files company-yasnippet)

Used backend: (company-capf :with company-dabbrev-code company-files)

Value of c-a-p-f: (lsp-completion-at-point codeium-completion-at-point tags-completion-at-point-function)
Major mode: terraform-mode
Prefix: ""
Completions: none

[dgutov]

I guess the backend "handled" things by offering no suggestions?

Indeed: "no suggestions" is also a valid result. Otherwise, if we performed fallback after fallback, we could reach per-word completions from dabbrev where there is no valid code suggestion (and actually shouldn't be). I'm not sure it's better, and it's somewhat more difficult to code up.

Anyway, it's good to know that company-files is still useful for your case. Actually, with this particular backend you could instead try putting it in front of company-capf instead of grouping them: (setq company-backends '(company-files company-capf)). Because company-files looks for ./ or / and returns nil otherwise, it should fall back to company-capf in other cases.

Unless terraform LSP sends some useful completions in those contexts too (those won't show up in this config).