Odd results with completion in a single file

[dgutov]

See this video: http://webmup.com/612b3/

1. "Filetype completion" doesn't work, probably because .h is not in SOURCE_EXTENSIONS, but nothing changed when I added it and restarted the server. Probably not a real issue, just something I don't understand.
2. The keyword completion behaves oddly, returning only some of the suggestions (as you can see, there are more, but it never returns enough for the scrollbar to appear).
3. Near the end of the video, with some tactical backspacing I've managed to make all completions start with ff. That must be a result of imperfect prefix-column adjustment.

[ptrv]

Regarding 1: You only get filetype completion after semantic triggers like ".", "->" or "::". In your example it's totally correct that you only get identifier completion. However, you can enable force semantic completion with C-c Y s in order to always trigger filetype completion.

[abingham]

I've also seen the problem where only a subset of the available completions are returned. It appears that we're displaying everything we get from ycmd, i.e. that ycmd is only returning a subset of the available completions. For example, I see this in the HTTP response from ycmd:

((completion_start_column . 5)
 (completions .
              [((insertion_text . "f1"))
               ((insertion_text . "f10"))
               ((insertion_text . "f11"))
               ((insertion_text . "f12"))
               ((insertion_text . "f13"))
               ((insertion_text . "f14"))
               ((insertion_text . "f15"))
               ((insertion_text . "f16"))
               ((insertion_text . "f17"))
               ((insertion_text . "f18"))]))

when there are actually hundreds of possible f1 completions.

[abingham]

@valloric Is there any reason (e.g. a configuration option) that ycmd would not be returning all possible completions? We've produced some (fairly pathological) examples where it appears that we're not receiving all of the options.

[dgutov]

you can enable force semantic completion with C-c Y s in order to always trigger filetype completion

That works fine, thanks. Kinda surprising to a new user, I'd say.

ycmd is only returning a subset of the available completions

Maybe it's trying to be clever and decides we don't need that many keyword completions? After all, there's usually a lot more of them than semantic ones.

[ptrv]

you can enable force semantic completion with C-c Y s in order to always trigger filetype completion

That works fine, thanks. Kinda surprising to a new user, I'd say.

AFAIK it's a feature of ycmd to have only identifier completion when not after a semantic trigger. Forcing semantic trigger can really slow completion down when using it in large projects. However, I find it useful for example when using C libraries with global functions when I need completion without a semantic trigger.

[Valloric]

@abingham For the identifier completer, there's a cap of max 10 completion strings returned. It needs to be there, otherwise ycmd can end up sending you a lot of text. We used to not have a cap, and then YCM would get all sad for some queries.

@dgutov As others have pointed out here, you really want to rely on the completion triggering logic in ycmd as-is. You get the identifier completer for "normal" typing where you're just typing an identifier whose name you know; it's super-fast and almost certain to return the completion string you want. When choosing a method on an object with . or ->, the semantic engine triggers because you might not be sure which method you need and thus you get "API exploration" support.

In the YCM (the Vim client for ycmd), you can also always force the semantic completion at any point with <ctrl-space> (I'm not sure how emacs-ycmd handles this). Forcing semantic completion all the time will just make you unhappy because libclang is usually not fast enough for that. This is why there's no option in YCM to always force semantic completion; it just leads to sadness. The identifier completer is seriously more than good enough for most completion needs.

This setup has been honed since ~2012 and works extremely well once you get used to it.

[dgutov]

We used to not have a cap, and then YCM would get all sad for some queries.

Maybe ycmd can take the cap value as a request parameter? Then emacs-ycmd could make it configurable. I'm sure 100 or even 1000 won't make a difference in terms of performance.

Forcing semantic completion all the time will just make you unhappy because libclang is usually not fast enough for that.

That's too bad. I'm probably not your target audience anyway (no C/C++ development), but if I got back into Java one day, I'd expect to see only semantic completions: local variables or imported static methods. Or class names when in fitting syntactic position. IOW, I often prefer "good" over "fast".

We might be used to projects of different magnitudes, though.

[ptrv]

Forcing semantic completion all the time will just make you unhappy because libclang is usually not fast enough for that.

That's too bad. I'm probably not your target audience anyway (no C/C++ development), but if I got back into Java one day, I'd expect to see only semantic completions: local variables or imported static methods. Or class names when in fitting syntactic position. IOW, I often prefer "good" over "fast".

I am using emacs-ycmd for C++ development and also on a quite large project, and I am pretty satisfied with the combination of identifier based and semantic completion.

We used to not have a cap, and then YCM would get all sad for some queries.

Maybe ycmd can take the cap value as a request parameter? Then emacs-ycmd could make it configurable. I'm sure 100 or even 1000 won't make a difference in terms of performance.

Would be a nice feature to being able to specify the cap to have more identifiers than 10.

@Valloric What number of identifiers in the result was the limit in your experience?

[Valloric]

Would be a nice feature to being able to specify the cap to have more identifiers than 10.

Yeah this makes sense. Open an issue on ycmd.

What number of identifiers in the result was the limit in your experience?

I don't remember. I know that not having a limit wasn't working out, and putting a limit of 10 seemed sensible. Nobody has complained about the limit until now. :)

That's too bad. I'm probably not your target audience anyway (no C/C++ development), but if I got back into Java one day, I'd expect to see only semantic completions: local variables or imported static methods. Or class names when in fitting syntactic position. IOW, I often prefer "good" over "fast".

For Java it would probably be a lot easier to have a performant semantic engine even for large codebases, since the language doesn't do imports with textual inclusion (#include) the way C & C++ do.

We might be used to projects of different magnitudes, though.

That's also a factor. At Google, I work in what's without a shadow of a doubt the largest single C++ codebase on the planet. That plus lots of generated protocol buffer headers leads to huuuuge translation units.

[ptrv]

@dgutov See #173

[dgutov]

@ptrv Thanks. FWIW, it doesn't seem to work even with updated emacs-ycmd and ycmd master, but maybe that's because python build.py doesn't do a full recompile.

Anyway, the main issue I'm currently worried about is the item 3, from the first message. I can still reproduce it after trying a bit.

[ptrv]

@dgutov The default is still 10. You probably have changed the variable ycmd-max-num-identifier-candidates, haven't you?

[dgutov]

I changed it, yes (to 20).

[abingham]

@dgutov When you set ycmd-max-num-identifier-candidates, did you do so after ycmd had been started? I looked at bit at the ycmd-max-num-identifier-candidates issue, and there's some surprising behavior here. The max-identifiers value is only used right when ycmd is started; we write it into a config file which ycmd reads and then deletes. That's just how ycmd works. If you need to change that value dynamically, the only easy way right now is to restart ycmd...ycmd-close ought to do the trick.

If you've set the value in your config and you're still not seeing the correct number of identifiers, then we probably have a bug.

As for your issue #3, I'm trying to reproduce it but haven't been able to. Are you still able to make it happen?

[dgutov]

Sorry, it doesn't seem to work at all now:

Contacting host: 127.0.0.1:45035
REQUEST [error] Error from parser json-read: (json-readtable-error)
REQUEST [error] Error (parse-error) while connecting to http://127.0.0.1:45035/event_notification.
previous-line: Beginning of buffer [2 times]
Contacting host: 127.0.0.1:45035
Company: An error occurred in auto-begin
Company: Back-end company-ycmd async timeout with args (candidates fo)
or: Cannot complete at point
REQUEST [error] Error from parser json-read: (json-readtable-error)
REQUEST [error] Error (parse-error) while connecting to http://127.0.0.1:45035/completions.
deferred error : (wrong-type-argument number-or-marker-p nil)

Any way I can ask the code to pop up an error backtrace?

[dgutov]

The max-identifiers value is only used right when ycmd is started; we write it into a config file which ycmd reads and then deletes. That's just how ycmd works.

I see. Personally, I'd expected it to be a request parameter (and there's no obvious mention that it isn't).

[abingham]

@dgutov Are you saying the emacs-ycmd doesn't work at all, i.e. from a fresh emacs restart? If so, are you sure you've got the most recent emacs-ycmd and ycmd?

[dgutov]

@abingham That is correct. Most recent Emacs, too.

But that also happens with Emacs 24.4, started with -Q, and with all packages installed from a blank slate.