Completing middle of word

[jojojames]

trafficstars

func application(application: UIApplication, didFinishLaunchingWithOptions launchOptions: [NSObject: AnyObject]?) -> Bool {
    // Override point for customization after application launch.
   NSOBjec
    return true
}

Typing NSObject doesn't trigger completion until NSObject<---

Anything before NSObjec <-- won't show completion but will show that there was a file written too.

[nathankot]

Yup, this is known: https://github.com/nathankot/company-sourcekit/issues/17#issuecomment-214588947 This was our attempt to support it: https://github.com/nathankot/company-sourcekit/blob/fa537304a0a6f90944d797ce58bc067603b6f987/company-sourcekit.el#L121-L131

But seems like sourcekit isn't working with what we give it. I think Xcode supports this type of completion by building an internal cache of symbols that sourcekit generates (by using the 400k+ response) - so instead of passing everything to sourcekit we intercept and look for existing symbols first.

We could actually use company-dabbrev in the middle to achieve a similar type of support...

[jojojames]

What's a little strange is merging company-sourcekit with dabbrev doesn't seem to get dabbrev completions (like sourcekit is blocking dabbrev's completion)

Company-diag returns Used backend: (company-sourcekit :with company-yasnippet company-dabbrev-code)

or this one

(company-sourcekit :with company-yasnippet company-dabbrev)

class SentBottlesController: UIViewController {

override func loadView() {
    self.view = SentBottlesView.init()
    Sent <-- won't complete SentBottlesView
}

[nathankot]

Hmmm the docs has this:

If the group contains keyword ‘:with’, the backends
listed after this keyword are ignored for the purpose of the ‘prefix’
command.

So supposedly dabbrev gets passed the same prefix as company-sourcekit generates, which most of the time wouldn't be compatible?

We could generate a different prefix inside company-sourcekit and pass it to dabbrev internally, and then merging the results - but not sure if semantically its a good idea for one completion method to contain another. And whether or not company-dabbrev is preferable to building an internal cache using the 400k+ sourcekit output.

Optimal solution at this point is to get company-sourcekit working with :with company-dabbrev

[algoterranean]

Hello all. I am looking into making an attempting on this mid-word completion feature and would like to get some feedback on the best approach. Is this a good spot for this discussion?

I was thinking:

1. Use sourcekittendaemon to get a list of Swift files in the project.
2. Run sourcekittendaemon /complete against each file in the project and compile the results into some structured object ala company-sourcekit--process-json. This should give a project-wide cache of all completion options, including what looks like tons of library-level information (that is, it's not just the entities in the .swift file)
3. Use these results (in particular the descriptionKey, typeName, and moduleName fields) to determine which results to populate for company.

Obviously there's more details involved, like caching the results, running it in the background without killing performance, and how exactly to determine the best results to display, but will this basic approach work?

And secondly, any suggestions on how to handle caching? I can roll my own but I noticed there's some discussion about using dabbrev? I haven't looked into dabbrev much yet so not sure how best to integrate that.

Thoughts?

[nathankot]

@algoterranean

Is this a good spot for this discussion?

Yup, here's good 😄 Great to hear that you're going to give this a crack!

1. Use sourcekittendaemon to get a list of Swift files in the project.
2. Run sourcekittendaemon /complete against each file in the project and compile the results into some structured object ala company-sourcekit--process-json. This should give a project-wide cache of all completion options, including what looks like tons of library-level information (that is, it's not just the entities in the .swift file)

May not be necessary to iterate through each file in the project since sourcekit essentially does that for you when you ask it for completion options in a buffer. Perhaps we can build a list of completions and cache them per-buffer, or incrementally building a project-wide cache of completions as new buffers are accessed. However from what I've seen the 400k+ response from sourcekit is more or less the same independent of which buffer you are referencing when making the call - so we could make the call simply on the current buffer, but update the project-wide cache of symbol completions.

These definitions would have to be updated though, Xcode probably has some sort of file watch that tells it when theres a throttled amount of change, and then updates the completion table on a background queue.

I can roll my own but I noticed there's some discussion about using dabbrev?

Don't think there would be a clean way to hook into dabbrev as it relies on completions being in the same buffer.

As for the caching, depending on the average size of these things we could simply try to keep it in memory. Or perhaps use something like redis with memory/disk as a fallback?

[algoterranean]

I spent the day looking into this, mainly focused on using the Xcode 8 beta and direct interop with /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/sourcekitd.framework/sourcekitd, in the hope that the new Xcode would include an updated SourceKit with better code completion.

I got the communication working fine (https://github.com/apple/swift/blob/master/tools/SourceKit/docs/Protocol.md#indexing) but code completion doesn't seem to be any different from 7.3. :(

However, I did notice something:

When attempting to query SourceKit for completion information on

import UIKit;UICol at offset 20, I got no result (using the source.request.codecomplete query).

When I ran the same against the source.request.indexsource query, however, I got back some data that at offset 16, it had recognized that UIColo was likely referencing a UIColor object:

Optional(["key.hash": "2ST469EV478F3", "key.entities": [["key.line": 1, "key.name": "UIColor", "key.usr": "c:objc(cs)UIColor", "key.column": 16]], "key.dependencies": ...

I will play around with this some more tomorrow and see if I can coax some more information out of it. I can post my code doing all this as well.

There is also some sort of "open" and "close" query syntax available that might allow the creation of sessions (meaning all indexing done under the same session key might be cumulative and improve over time or something... maybe). Not sure how it actually works yet though. See https://github.com/apple/swift/tree/82509cbd7451e72fb99d22556ad259ceb335cb1f/test/SourceKit/CodeComplete/Inputs/custom-completion for some examples.

[nathankot]

Thanks for spending time on this @algoterranean! https://github.com/apple/swift/blob/master/tools/SourceKit/docs/Protocol.md is actually really insightful, I'm glad that this has been made available and thanks for bringing it up :)

---

When I ran the same against the source.request.indexsource query, however, I got back some data that at offset 16, it had recognized that UIColo was likely referencing a UIColor object:

I believe this is expected behaviour, the Index request (aka 400k+ ?) is responsible for building a list of symbols used for dabbrev-like completion, and the Code Completion request is responsible for things like method completion. It'd be nice if sourcekit could merge these responsibilities into complete, but I guess there were reasons for this separation.

Although I'm at a loss as to why we are getting index-like responses from sourcekittendaemon, I don't believe it ever sends source.request.indexsource to sourcekit..

---

There is also some sort of "open" and "close" query syntax available that might allow the creation of sessions (meaning all indexing done under the same session key might be cumulative and improve over time or something... maybe). Not sure how it actually works yet though. See https://github.com/apple/swift/tree/82509cbd7451e72fb99d22556ad259ceb335cb1f/test/SourceKit/CodeComplete/Inputs/custom-completion for some examples.

The concept of sessions sounds nice, although I can't seem to understand anything from those examples. Could you clarify a bit more?