`sourcekitten index` always fails, `complete` only returns stdlib completions

[bmhatfield]

I have tried with a lot of different swift files, and I always get back the same error:

[bhatfield Jobs]% sourcekitten index --file Job.swift
The operation couldn’t be completed. (SourceKittenFramework.Request.Error error 2.)

sourcekitten complete never returns completion information for my files, only stdlib types and keywords. I expect it to include information about the types in the files I point it to.

SourceKitten version:

[bhatfield Jobs]% brew info sourcekitten
sourcekitten: stable 0.21.1 (bottled), HEAD

Xcode version:

[bhatfield Jobs]% xcodebuild -version
Xcode 9.4.1
Build version 9F2000

Swift version:

[bhatfield Jobs]% swift --version
Apple Swift version 4.1.2 (swiftlang-902.0.54 clang-902.0.39.2)
Target: x86_64-apple-darwin17.7.0

How can I get more debug information or otherwise find resources to help me resolve this issue?

[jpsim]

SourceKit needs all compiler arguments for a Swift file to be indexed properly. Here's an example:

$ sourcekitten index \
  --file "$(pwd)/Source/sourcekitten/main.swift" \
  -- \
  -sdk "$(xcrun --show-sdk-path)" \
  -j4 "$(pwd)/Source/sourcekitten/main.swift"

Output

{
  "key.dependencies" : [
    {
      "key.filepath" : "\/Applications\/Xcode-10.1.app\/Contents\/Developer\/Toolchains\/XcodeDefault.xctoolchain\/usr\/lib\/swift\/macosx\/x86_64\/Swift.swiftmodule",
      "key.hash" : "358PQT77M2I60",
      "key.is_system" : true,
      "key.kind" : "source.lang.swift.import.module.swift",
      "key.name" : "Swift"
    },
    {
      "key.dependencies" : [
        {
          "key.filepath" : "\/var\/folders\/8_\/sm32hcd162v1dm0fw00gyrph0000gn\/C\/org.llvm.clang.jsimard\/ModuleCache\/AL6JH58ZYTUK\/Darwin-3L4TB4EN7ZT10.pcm",
          "key.is_system" : true,
          "key.kind" : "source.lang.swift.import.module.clang",
          "key.name" : "Darwin"
        },
        {
          "key.filepath" : "\/Applications\/Xcode-10.1.app\/Contents\/Developer\/Toolchains\/XcodeDefault.xctoolchain\/usr\/lib\/swift\/macosx\/x86_64\/Swift.swiftmodule",
          "key.hash" : "358PQT77M2I60",
          "key.is_system" : true,
          "key.kind" : "source.lang.swift.import.module.swift",
          "key.name" : "Swift"
        },
        {
          "key.filepath" : "\/var\/folders\/8_\/sm32hcd162v1dm0fw00gyrph0000gn\/C\/org.llvm.clang.jsimard\/ModuleCache\/AL6JH58ZYTUK\/SwiftShims-2QHGN5QHW6J6K.pcm",
          "key.is_system" : true,
          "key.kind" : "source.lang.swift.import.module.clang",
          "key.name" : "SwiftShims"
        }
      ],
      "key.filepath" : "\/Applications\/Xcode-10.1.app\/Contents\/Developer\/Toolchains\/XcodeDefault.xctoolchain\/usr\/lib\/swift\/macosx\/x86_64\/Darwin.swiftmodule",
      "key.hash" : "2ML5GXDSA1CEM",
      "key.is_system" : true,
      "key.kind" : "source.lang.swift.import.module.swift",
      "key.name" : "Darwin"
    },
    {
      "key.dependencies" : [
        {
          "key.filepath" : "\/var\/folders\/8_\/sm32hcd162v1dm0fw00gyrph0000gn\/C\/org.llvm.clang.jsimard\/ModuleCache\/AL6JH58ZYTUK\/Dispatch-X6ASN9ZP3D0Q.pcm",
          "key.is_system" : true,
          "key.kind" : "source.lang.swift.import.module.clang",
          "key.name" : "Dispatch"
        },
        {
          "key.filepath" : "\/Applications\/Xcode-10.1.app\/Contents\/Developer\/Toolchains\/XcodeDefault.xctoolchain\/usr\/lib\/swift\/macosx\/x86_64\/Swift.swiftmodule",
          "key.hash" : "358PQT77M2I60",
          "key.is_system" : true,
          "key.kind" : "source.lang.swift.import.module.swift",
          "key.name" : "Swift"
        },
        {
          "key.filepath" : "\/var\/folders\/8_\/sm32hcd162v1dm0fw00gyrph0000gn\/C\/org.llvm.clang.jsimard\/ModuleCache\/AL6JH58ZYTUK\/_SwiftDispatchOverlayShims-2QHGN5QHW6J6K.pcm",
          "key.is_system" : true,
          "key.kind" : "source.lang.swift.import.module.clang",
          "key.name" : "_SwiftDispatchOverlayShims"
        }
      ],
      "key.filepath" : "\/Applications\/Xcode-10.1.app\/Contents\/Developer\/Toolchains\/XcodeDefault.xctoolchain\/usr\/lib\/swift\/macosx\/x86_64\/Dispatch.swiftmodule",
      "key.hash" : "1JXLH637WAS8P",
      "key.is_system" : true,
      "key.kind" : "source.lang.swift.import.module.swift",
      "key.name" : "Dispatch"
    }
  ],
  "key.entities" : [
    {
      "key.column" : 1,
      "key.kind" : "source.lang.swift.ref.class",
      "key.line" : 21,
      "key.name" : "DispatchQueue",
      "key.usr" : "c:objc(cs)OS_dispatch_queue"
    },
    {
      "key.column" : 15,
      "key.kind" : "source.lang.swift.ref.function.method.class",
      "key.line" : 21,
      "key.name" : "global(qos:)",
      "key.receiver_usr" : "c:objc(cs)OS_dispatch_queue",
      "key.usr" : "s:So17OS_dispatch_queueC8DispatchE6global3qosAbC0D3QoSV0G6SClassO_tFZ"
    },
    {
      "key.column" : 28,
      "key.kind" : "source.lang.swift.ref.enumelement",
      "key.line" : 21,
      "key.name" : "default",
      "key.usr" : "s:8Dispatch0A3QoSV0B6SClassO7defaultyA2EmF"
    },
    {
      "key.column" : 37,
      "key.is_dynamic" : true,
      "key.kind" : "source.lang.swift.ref.function.method.instance",
      "key.line" : 21,
      "key.name" : "async(group:qos:flags:execute:)",
      "key.receiver_usr" : "c:objc(cs)OS_dispatch_queue",
      "key.usr" : "s:So17OS_dispatch_queueC8DispatchE5async5group3qos5flags7executeySo0a1_b1_F0CSg_AC0D3QoSVAC0D13WorkItemFlagsVyyXBtF"
    },
    {
      "key.column" : 9,
      "key.kind" : "source.lang.swift.ref.function.free",
      "key.line" : 35,
      "key.name" : "fputs(_:_:)",
      "key.usr" : "c:@F@fputs"
    },
    {
      "key.column" : 29,
      "key.entities" : [
        {
          "key.column" : 29,
          "key.kind" : "source.lang.swift.ref.function.accessor.getter",
          "key.line" : 35,
          "key.usr" : "s:6Darwin6stderrSpySo7__sFILEVGvg"
        }
      ],
      "key.kind" : "source.lang.swift.ref.var.global",
      "key.line" : 35,
      "key.name" : "stderr",
      "key.usr" : "s:6Darwin6stderrSpySo7__sFILEVGvp"
    },
    {
      "key.column" : 1,
      "key.kind" : "source.lang.swift.ref.function.free",
      "key.line" : 39,
      "key.name" : "dispatchMain()",
      "key.usr" : "c:@F@dispatch_main"
    }
  ],
  "key.hash" : "14NUQH5RGOSJ4"
}

---

sourcekitten complete never returns completion information for my files, only stdlib types and keywords. I expect it to include information about the types in the files I point it to.

I don't have much real world experience using SourceKit's code completion request and honestly the way SourceKit handles code completion is hard to follow. My general suggestion for getting more debug info and finding other resources should help.

---

Generally if I want to do something with SourceKit that Xcode already does, I launch Xcode with SOURCEKIT_LOGGING=3 in its environment and try to find relevant SourceKit requests to then recreate those using SourceKitten. For example:

$ SOURCEKIT_LOGGING=3 /Applications/Xcode-10.1.app/Contents/MacOS/Xcode

This logging is really verbose but odds are if there's anything Xcode can do with Swift, you can find out how to do it using SourceKit(ten) yourself. Sometimes it's pretty complicated though, such as appears to be the case with code completion.

[gubikmic]

It seems like this stopped working. The above example now yields an error (The operation couldn’t be completed. (SourceKittenFramework.Request.Error error 2.)) when run in the SourceKitten folder.

[jpsim]

This is exercised by integration tests, and those are still passing, so you'll need to share specific ways in which you're running this, against what tooling versions, and the ways in which it's failing.

[gubikmic]

I get a bunch of failing tests, including testIndex(). Maybe I'm missing something - is this compatible to use in Xcode 11 beta?

[jpsim]

It's quite likely that this needs to be updated as Xcode betas come out, yes.

[huy-le]

Hi @jpsim, thanks for suggestion to using SOURCEKITLOGGING=3. I have used it to debug the Xcode and found that they have interested request called key.request: source.request.document.genius-callers-callees . However, it isn't declared in Sourcekitd and I cannot call it via yaml. As your experience, do you know whether Apple are using a private SourceKit that include some private requests that not include in the Sourcekit open source project?

[jpsim]

As your experience, do you know whether Apple are using a private SourceKit that include some private requests that not include in the Sourcekit open source project?

Unfortunately yes, I can confirm that they do this. I believe it's called "SourceKitClient".

[huy-le]

Thanks @jpsim, do you know is it possible to link sourcekitten with that SourceKitClient?

[jpsim]

I’ve tried to interface with SourceKitClient a few times probably around ~2+ years ago at this point but never got it to work.

[huy-le]

Thanks a lot @jpsim! I will have a look on it to see whether I get any luck.

[danowensdev]

Hi @huy-le, did you make progress here? :)