How to trace/debug auto import suggestions?

[jedwards1211]

Acknowledgement

• [x] I acknowledge that issues using this template may be closed without further explanation at the maintainer's discretion.

Comment

I'm authoring a package with

  "type": "module",
  "exports": {
    "./package.json": "./package.json",
    "./*": {
      "types": {
        "import": "./dist/*.d.ts",
        "default": "./dist/*.d.cts"
      },
      "import": "./dist/*.js",
      "default": "./dist/*.cjs"
    }
  },

When I install it in a new project I can't get tsserver to auto suggest any imports from these files (until I've manually added an import from a given file), regardless of module and moduleResolution settings.

How do I turn on debug logging that will help me figure out why tsserver isn't offering any auto import suggestions from my package?

I tried turning on "typescript.tsserver.log": "verbose" and looking at those logs but I don't see anything that sheds light on how tsserver is making decisions.

Is there any logging option for this? I've used traceResolution to debug module resolution issues that cause compile errors and it was super helpful, but import suggestions still seem like a black box.

[RyanCavanaugh]

There isn't a huge amount of logging in this code (see src/server/project.ts around line 2500 for the AutoImportProviderProject).

Given that package.json I'm guessing you have more than 10 entry points and are hitting the built-in perf limiter that turns off package enumeration. Have you already set the "Include package json auto imports" option to 'on'?

[jedwards1211]

Thanks for the tip, I figured there wasn't much public documentation of this.

Yes, I have more than 10 entry points, one per utility function so that I don't have to use any bundler to minimize the memory usage/startup time when importing in Node. I tried setting "Include package json auto imports" option to 'on' but unfortunately it didn't make a difference, eventually I'll see if I can attach a debugger to tsserver.

I'm seeing this same issue with the popular es-toolkit project too, a popular lodash replacement. Also, MUI advises people to configure TypeScript not to suggest top-level imports from their packages and import from their fine-grained entrypoints (of which there are hundreds) instead.

Obviously it would take a bit more startup work, but I think it would make more sense to limit based upon the total size of a package, or modules scanned thus far while building an index of suggestions, than the number of entrypoints, because a single entry point can be enormous (e.g. the old aws-sdk) and a package with hundreds of fine-grained entry points could be quite small in comparison.

[RyanCavanaugh]

Totally. We're rewriting auto-imports in TS7 and it'll have smarter bail-out heuristics.

If you can publish your package somewhere we can pull it, we could probably see why it's not getting picked up. Checking in tsgo would be even better, though.

[jedwards1211]

Okay, here's my package: https://www.npmjs.com/package/@jcoreio/utils?activeTab=code

Unlike many packages you'll see, I'm including CJS, ESM, source and declaration maps for both, and original source files in the published package, so that Jump to Definition brings up readable source code instead of just declaration files. I don't know if having all of those files somehow trips tsserver up?

I haven't gotten my feet wet with tsgo yet, need to try that

[andrewbranch]

Related: https://github.com/microsoft/TypeScript/issues/60704

I'm reworking the backend of auto-imports in tsgo right now to avoid all the issues mentioned there.

[jedwards1211]

@andrewbranch if it still bails out anywhere after your changes, could you at least make it log where and why it bailed out? (if tsgo doesn't already, I haven't tried tsgo yet)

[andrewbranch]

There's no bail-out in my new version; the design is totally different such that that wouldn’t make sense / be possible.

[jedwards1211]

Oh I see, do you mean if there are too many dependencies for all auto imports to fit in memory at once it will dynamically look things up? I assumed there was still going to be some kind of bail-out to prevent OOM if the number of deps is huge

[andrewbranch]

The reason it's not possible to do the same thing in the rewrite is that instead of having a bucket of auto-imports sourced from your Program (which includes all node_modules files you already transitively import/reference in any way), and then another bucket of auto-imports of all the node_modules files we missed (you know, as if we tacked that feature on as a response to user feedback years after auto-imports was designed), the new system will have a bucket of auto-imports from your non-node_modules project files, and then one bucket of node_modules files for every relevant node_modules directory. We won’t bail out of searching for extra dependencies, because there’s no special distinction between the files that are already in your program and the ones that aren’t anymore. That turns out to make a lot of stuff way easier and way cheaper, and also better matches users’ intuition of what auto-imports should care about.

The old method required creating a Program in order to discover and hold all the extra files for auto-imports. ASTs are they heavyweights in terms of memory usage. The new system won’t need to retain any ASTs; all the relevant info is extracted from the ASTs and then we can throw them away. I doubt OOMing will be a concern for projects of any size, but if we do end up needing some way of collecting fewer exports, it will probably be on an axis that’s not “in my program” vs. “not in my program yet.” The lowest hanging fruit is searching only types/main/exports and not walking package directories for other files when exports isn’t present.

[jedwards1211]

A long time ago I made an auto import server that worked with Flow, and maybe I was accidentally hanging onto something I shouldn't, but I was surprised how high the memory usage could get from ostensibly just the index of stuff in memory.

One thing that might have added to it is, I didn't just index exports, I also indexed all of the import statements in project files to add extra smarts like sorting multiple suggestions for the same name by usage count, and to be able to suggest import { foo as bar } from 'foo' for bar if that import statement existed in another project file.