Go To Source Definition feedback thread

[andrewbranch]

Seeding a thread for feedback on / issues with Go To Source Definition (#48264) ahead of the 4.7 release.

TL;DR

TypeScript 4.7 (already in nightly) and VS Code 1.67 (already released) will contain a new navigation command called “Go To Source Definition” that attempts to find a definition in non-ambient TypeScript or JavaScript code. (Ambient code means .d.ts files or declare contexts inside regular TS files—in other words, definitions separated from concrete implementation.) The most common reason to use this is if you want to look at the JS implementation of something you imported from node_modules, where Go To Definition only jumps to .d.ts files.

Our ability to find concrete definitions in JS when Go To Definition returns only .d.ts files varies wildly based on a lot of complicated stuff. Sometimes this feature returns results that we can be nearly 100% confident in. Sometimes it returns a guess that definitely might be wrong. Sometimes it returns nothing. If it does something that surprises you, I would like to hear about it here, even though it might be a known limitation. Make sure to include:

• The code you triggered the request on
• The import statement importing the thing you tried to navigate to
• What result(s) were returned, if any (screenshot is ok for these top 3)
• What version of the library you tried to navigate to is in your node_modules (check its package.json)
• If you had a corresponding @types library in your node_modules, and what version it is

Known limitations

• The main thing that makes this feature go from 100% confidence to guesses that could go wrong is JavaScript whose exports we can’t analyze, probably because they have some kind of UMD wrapper. When that happens, we try guesses. The quality/presence of guesses is impacted by:

  • How closely the directory structure of a @types package, if present, matches that of its JS counterpart
  • Whether declarations in a single JS file share the same name with others at the same level of nesting. For example, import { add } from "lodash" currently returns three results; two refer to the correct function, but one is a false positive (SetCache.prototype.add):
  • 

• Function parameters, and properties accessed on them, are unlikely to return any results, because definition information doesn’t tell us every caller of the function—we only know where the parameter itself is declared, and what type it expects. The values passed into it could come from anywhere, even if in practice they have a single definition provided by a library:

  import yargs from "yargs";
  //     ^ works here
  yargs(process.argv.slice(2))
    .command(
  // ^ works here
      "$0 <traceDir>",
      yargs => yargs.positional("traceDir", {
  //           ^ goes to arrow function param
  //                 ^ doesn't find anything!
  

FAQ and Q that should be FA but aren’t

Can you please make this the default / give me an option to make this the default for Go To Definition / ⌘-click?

Not right now, but comment / 👍 an existing relevant comment if you feel strongly about this and tell me a bit about why.

Can I make a keyboard shortcut for this?

Yes. Command palette → Preferences: Open Keyboard Shortcuts (JSON)

{
  "key": "cmd+shift+F12",
  "command": "typescript.goToSourceDefinition",
  "when": "editorTextFocus"
}

What’s the difference between this and Go To Implementations?

Go To Implementations is an interesting algorithm that does a lot of different things and is a bit hard to explain holistically. The two commands are similar in that they both avoid returning ambient results. However, Go To Implementations uses Find All References under the hood to (among other things) search for values that satisfy some type. Go To Source Definition mostly uses Go To Definition under the hood, which traces variables/properties to their declarations and through imports/exports. It does not try to find the origin of every possible value that might come to inhabit that variable/property. The thing that makes Go To Source Definition different from every other command is that if Go To Definition returns only results in .d.ts files, it will try again with a different module resolver that completely ignores .d.ts files, allowing it to find JS files that are otherwise “shadowed” by .d.ts files in your actual compilation.

I’m a DefinitelyTyped maintainer. What can I do to increase the chances Go To Source Definition works on exports from my definitions?

Ensure the directory structure of your definitions is identical to that of the JS library you’re typing (at least as far as the public API goes). This means don’t move types out of index.d.ts into a helpers.d.ts just for code organization—helpers.d.ts should only exist if it types a helpers.js file. Conversely, if the JS library is broken up into many files that all get re-exported by the index/main file, declare your types in these files and re-export them too. (This is the correct way to author DT libraries anyway; Go To Source Definition compatibility is just a secondary benefit.)

I’m a library author who ships my own types. What can I do to make Go To Source Definition work on exports from my library?

Consider if it’s appropriate to ship your TypeScript sources to npm. If you do, make sure you compile with "declarationMap": true and ship your .d.ts.map files to npm too. When you do that, Go To Definition will already navigate to your .ts sources, making Go To Source Definition unnecessary (it simply executes Go To Definition when this happens).

[DanielRosenwasser]

I was surprised that this didn't work as cleanly with a module.exports = someFunc, like in left-pad.

// index.mts
import leftPad = require("left-pad");

leftPad("", 4);

// tsconfig.json
{
    "compilerOptions": {
        "module": "node16",
        "strict": true,
        "outDir": "./out",

    }
}

Screenshot

[DanielRosenwasser]

Found a crash in an inferred JS project when using a destructured declaration on a require.

// index.js
const { ncp } = require("ncp");

// package.json
{
  "name": "yadda",
  "version": "1.0.0",
  "description": "Yadda",
  "main": "index.js",
  "dependencies": {
    "ncp": "^2.0.0"
  }
}

TSServer Crash

<semantic> Response received: findSourceDefinition (625). Request took 3 ms. Success: false . Message: Error processing request. Debug Failure. Illegal value: SyntaxKind: BindingElement
Error: Debug Failure. Illegal value: SyntaxKind: BindingElement
    at Object.tryGetModuleSpecifierFromDeclaration (ms-vscode.vscode-typescript-next-4.8.20220524/node_modules/typescript/lib/tsserver.js:16587:26)
    at Object.getDefinitionAtPosition (ms-vscode.vscode-typescript-next-4.8.20220524/node_modules/typescript/lib/tsserver.js:139272:63)
    at Object.getDefinitionAtPosition (ms-vscode.vscode-typescript-next-4.8.20220524/node_modules/typescript/lib/tsserver.js:164900:38)
    at IpcIOSession.Session.findSourceDefinition (ms-vscode.vscode-typescript-next-4.8.20220524/node_modules/typescript/lib/tsserver.js:176077:50)
    at Session.handlers.ts.Map.ts.getEntries._a.<computed> (ms-vscode.vscode-typescript-next-4.8.20220524/node_modules/typescript/lib/tsserver.js:175294:61)
    at ms-vscode.vscode-typescript-next-4.8.20220524/node_modules/typescript/lib/tsserver.js:177441:88
    at IpcIOSession.Session.executeWithRequestId (ms-vscode.vscode-typescript-next-4.8.20220524/node_modules/typescript/lib/tsserver.js:177432:28)
    at IpcIOSession.Session.executeCommand (ms-vscode.vscode-typescript-next-4.8.20220524/node_modules/typescript/lib/tsserver.js:177441:33)
    at IpcIOSession.Session.onMessage (ms-vscode.vscode-typescript-next-4.8.20220524/node_modules/typescript/lib/tsserver.js:177467:35)
    at process.<anonymous> (ms-vscode.vscode-typescript-next-4.8.20220524/node_modules/typescript/lib/tsserver.js:180102:31)
    at process.emit (node:events:390:28)
    at emit (node:internal/child_process:917:12)
    at processTicksAndRejections (node:internal/process/task_queues:84:21)

[andrewbranch]

@DanielRosenwasser can you give me compiler options for that? Also, does go-to-def jump to your ATA cache?

[andrewbranch]

inferred JS project

Oh, missed this. ~~But still can’t reproduce 🤔~~ Was able to reverse engineer from the stack.

[scttcper]

would enabling "declarationMap": true mean we need to include the original .ts files as well? I don't think many packages currently do this. But this does work with @types/ typed packages? Is there any way to enable this for packages that ship their types but not the original .ts files?

[andrewbranch]

would enabling "declarationMap": true mean we need to include the original .ts files as well?

Yes.

But this does work with @types/ typed packages? Is there any way to enable this for packages that ship their types but not the original .ts files?

What is “this” in this context? declarationMap maps locations from .d.ts files to the .ts file that created them. I don’t quite understand the question/suggestion for what you could do with this in the absence of the TS sources.

And to be clear, declaration maps have always been used for Go To Definition when available—nothing new about that in 4.7. rxjs is an example in the wild—a plain old Go To Definition on anything you import from there will go to the TS source.

[scttcper]

I was confused because the packages I was trying with "go to source definition" weren't working. I think this is because I had "moduleResolution": "node16", in my tsconfig. Its fixed with typescript 4.7.3

[jasonwilliams]

Hey @andrewbranch great work on this, I had a quick Q (you kind of mentioned it in your original comment).

Shouldn't "Go To Definition" be the default for this and there should be a separate command for going to the type definition? I only ask this because it will be inconsistent with other languages being used in VSCode where "Go To Definition" would go to the original source definition of that symbol. I feel like most people using the "Go To Definition" command are expecting it to go to the source code not the type, so I was surprised to see the solution was not to change it but instead add a new command.

This will end up making TS/JS a "special snowflake" in the VSCode UI where you need to navigate to another command to get the same result that you'd have from other languages. I could be wrong but that seems to go against VSCode's goal to have familiar UX across all languages

[andrewbranch]

I think I answered that in this Twitter thread. Let me know if that leaves you with any more questions.

[jasonwilliams]

Ok thanks @andrewbranch I understand this is not wanting to change/break workflow for those who are already used to it.

[andrewbranch]

Well there’s also the fact that Go To Type Definition means something completely different, and Go To Definition is kind of sacred in that it shows you with 100% confidence what the compiler thinks the definition of some code is. Keep in mind that Go To Definition does take you to the original, concrete source when it can do so with 100% confidence—that is, when declaration maps and original TS sources exist. It would be great if this were always true, but the reality is that TS is a bit of a special snowflake in that our “header files” and “implementation files,” if you want to extend the analogy to other languages, are often written, by hand, by different parties at different times under different versions installed to different locations and the extent to which they match each other is determined solely by the skill of the declaration author and the ability of the user to install matching versions of each (I’m referring to getting .d.ts files from DefinitelyTyped, of course). Such uncertainty deserves to be treated differently than the distinction between a header and implementation file compiled from a single source. And also/again, types are so central to the process of writing TypeScript that even if we could always take you to the right spot in the JS with 100% certainty, I still don’t think we’d call it the “definition” (at least for TS users).

[jzabinski-dolios]

When using this feature for an imported Typescript library (AngularFirestore), in VSCode v1.68 and Typescript 4.5.5, I get 'No source definitions found'. I don't know why; my guess is that this feature requires Typescript 4.7.

If my guess is right: could we have a better message please? Maybe 'Typescript 4.7 required in order to support Go To Source Definition'?

Or probably better: disable the feature until the environment can support it?

[andrewbranch]

@jzabinski-dolios VS Code does hide the menu item if it didn’t see TypeScript 4.7 in the workspace, and gives this error message if you invoke it by keyboard shortcut anyway:

Unless something really weird is happening, I would guess you’re actually using VS Code’s bundled version of TypeScript (now 4.7) in the workspace, and we really couldn’t find the source for that import due to one of the limitations described in the issue body.

[danieliser]

Glad to see this finally made it in. Could we get an option to exclude folders from the search?

Specifically mine keeps linking to /build/ files, when /src/ files are there in original JSX form.

Would love to exclude ./build/, ./dist/ for example from the search of the package.

This should both help me narrow my own needs, but also make it more performant.

[andrewbranch]

@danieliser which package is doing that? Can you provide a repro? The answer to your question is very likely “no.” We found dist because that’s where the package.json pointed. If we exclude dist, we’ll just find nothing instead. How would you propose this work? Should we simply assume src is equivalent to dist by convention? 😬 What if that’s not true?

One neat example of how this can work well is React. Its entry point sets module.exports to either the minified or dev version of the source based on process.env.NODE_ENV. Because of that, we automatically return results both in the minified and the non-minified source.

[ThomasSteinbach]

It would be nice if I could tell VSCode where the source to go to is.

Currently VSCode jumps to the installed and transpiled TypeScript package. The caveat is that you could not fully understand neither the .d.ts nor the .js files.

However I have clone the original source of the installed package on my disk. So I could tell VSCode where to go to instead and browse the original TypeScript files.

I know a similar feature (~15yrs ago) from Eclipse, where you could tell the IDE where to find the source of a Java package. Instead going to the compiled sources, Eclipse jumped to files in the package.

[benhickson]

What's the best way to jump to the typescript definition, the .ts file?

We're in a typescript monorepo, and cmd-clicking on imports between packages in the same monorepo results in the .d.ts file, and this new function results in the .js file... however I want to get to the original .ts file.

[RyanCavanaugh]

@benhickson you need to enable declaration source maps for that to happen

[Zamiell]

Today, I tried out the "Go to Source Definition" feature with my NX monorepo. (NX is quickly becoming one of the most popular TypeScript monorepo tools.)

In an NX monorepo, the dist folder for all of the subpackages lives in the monorepo root, like this:

monorepo/
├── packages/
|   ├── foo/
|   └── bar/
└── dist/
    └── packages/
        ├── foo/
        └── bar/

This is a break from the more conventional monorepo layout, where each dist folder resides in the subpackage directory itself, like this:

monorepo/
└── packages/
    ├── foo/
    |    ├── src/
    |    └── dist/
    └── bar/
         ├── src/
         └── dist/

NX changes this convention for a good reason - so that the dist directory can be cached.

Unfortunately, the consequence of this change is that the declarationMap compiler option that is recommended in the OP does not work, because the mapping will go upwards into parent directories that do not exist on end-user machines once the package is uploaded to npm. (npm packages are uploaded from the "dist" directory.)

~~Fortunately, there is an alternate option: the "Go to Source Definition" feature will also work if the source file foo.ts exists directly next to the foo.js file - no mapping file needed. So, NX users can manually copy the .ts files into place directly beside their .js/.d.ts counterparts using a custom build script. (Right now, I'm using Bash with cp -RT "$DIR/src" "$OUT_DIR/".)~~ (Turns out this doesn't work at all, so I'm not sure what the actual solution is for NX users, if any.)

With that said, manually merging source directories with dist directories in a build script is a little clunky, and requires technical knowledge of how all of these working parts plug together. A much better solution would be a compiler flag.

Currently, TypeScript offers the "declaration": true compiler flag, which handily creates .d.ts files next to the compiled .js files. I think it is reasonable for there also to exist a "sourceFiles": true compiler flag, which would handily create/copy the .ts files next to the .js and .d.ts files. Turning this option on would make the "Go to Source Definition" feature just work (tm) for everyone, regardless of what directory structure they happen to be using, regardless of whether they are using source maps, regardless of whether they have remembered to include the src directory inside of the "package.json" files directive, etc.

This feature is kind-of-related to #30835, which is closed as wont-fix. With that said, I think that if the TypeScript team is going to introduce "Go to Source Definition" as an official TypeScript feature, it would be really great to also offer a solution to make it work with minimal hassle for everyone.

[andrewbranch]

Unfortunately, the consequence of this change is that the declarationMap compiler option that is recommended in the OP does not work, because the mapping will go upwards into parent directories that do not exist on end-user machines.

Sounds like this is just a bug with declarationMap, a project misconfiguration, or a weird interaction between tsc and nx. There’s no reason Go To Source Definition should need to be involved to support this use case; if we are the ones building all your code, of course we should be able to seamlessly map between our own inputs and outputs. Can you file a new bug with a small repro of this?

[Zamiell]

Andrew,

In this case, TypeScript can't map to the source files, because they don't exist. See the directory structure map that I sketched out above.

For example, say that I run npx tsc in the following folder:

/Users/Zamiell/monorepo/packages/foo

It creates the following files:

/Users/Zamiell/monorepo/dist/packages/foo/index.d.ts
/Users/Zamiell/monorepo/dist/packages/foo/index.d.ts.map
/Users/Zamiell/monorepo/dist/packages/foo/index.js

The contents of index.d.ts.map are:

{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../../packages/foo/src/index.ts"]}

In this case, the root directory of the uploaded files to npm are /Users/Zamiell/monorepo/dist/packages/foo, so the "sources" property looks backwards into directories that don't exist.

• If I manually add the .ts files after the compilation step, that doesn't help anything, because the .d.ts.map files are already created.
• If I manually add the .ts files before the compilation, that doesn't help anything, because then all of the import statements break, as the paths to all of the libraries and what-not have changed.

Let me know if that isn't clear and I can make an example repo.

[andrewbranch]

Oh, I thought you were referring to your local development workflow with nx, not consuming a package published to npm that was written with nx. I understand now.

[Zamiell]

By the way, I don't think the problem that I am outlining is exclusive to NX monorepos. Here are two more scenarios that could be relatively common:

Scenario A

• Alice has a npm library written in TypeScript.
• Alice follows npm best practices by using the "files" directive inside of her "package.json" file as a whitelist to only upload specific files to npm.
• Alice decides to try out the new "Go to Source Definition" feature. No matter which compiler knobs she twists, she just can't get it to work.
• After some hours of troubleshooting, Alice realizes that she has to actually add her whole "src" folder to the "files" directive inside of her "package.json" file so that the maps will work properly in end-user code.
• In retrospect, this was obvious, but it was an annoying footgun that wasn't covered in any of the documentation. All Alice wanted to do was to try out this new feature, and she had to go down the rabbit hole of troubleshooting source maps and package.json configuration. If there would have simply been instructions to "just turn on the sourceFiles compiler flag", it would have been a completely seamless process for Alice.

Scenario B

• Bob has an npm library written in TypeScript.
• Similar to Alice, Bob uses the "files" directive inside of his "package.json" file. But unlike Alice, he has a specific reason to do so: there are several artifacts within the src subdirective that must never be published to npm.
• Bob decides to try out the new "Go to Source Definition" feature, but it's unclear for him how to get his TypeScript files to npm.
• Bob can't simply add the "src" directory to the "package.json" file, because he doesn't want to include the artifacts. Furthermore, he can't do a glob like "src/**/*.ts", because that would include build scripts and other sensitive files.
• Instead, Bob is forced to copy-paste the N globs from "tsconfig.json" to "package.json", which kick-starts everything into working properly.
• Later on, Bob updates his "tsconfig.json" file to include more files, but forgets that because of this janky system, the configuration information actually resides in two places instead of one place.
• Bob spends hours troubleshooting why the "Go to Source Definition" feature stops working for a specific subset of his library, eventually finding out that the "package.json" globs forgot to get updated.
• In retrospect, this was obvious, but it was an annoying footgun because having configuration information in two places instead of one violates DRY. If there would have simply been instructions to "just turn on the sourceFiles compiler flag", it would have been a completely seamless process for Bob.

[andrewbranch]

Three things:

1. I’m totally sympathetic to this problem; it seems tricky.
2. Copying .ts files next to output .d.ts files can never ever happen. The .ts files have a higher priority for module resolution, so anyone consuming such a directory is going to be in an extremely bad situation. This would wreck performance and break project references beyond all recognition. There is no way to get around this that would be remotely backward-compatible. All for brainstorming solutions but this one is a dead end, full stop.
3. To be totally clear, the behavior of using declaration maps to map Go To Definition from .d.ts to .ts files is not new. This has been a feature pretty much forever and is totally unrelated to Go To Source Definition beyond the fact that Go To Source Definition will simply defer to Go To Definition any time Go To Definition provides a suitable result. I mentioned it in the issue body here only because it’s something lots of folks don’t know and as a library author, might be thinking about only because they heard about Go To Source Definition. Accordingly, I’d like to move this discussion elsewhere—this thread is intended to track bugs and limitations of this new feature, not help library authors configure their publishing process to take advantage of an old one.

[Zamiell]

This error message isn't very helpful. Can we add to the message all of the paths that were tried? It would make debugging this feature much less painful.

[andrewbranch]

The plan is actually to make that message less prominent: https://github.com/microsoft/vscode/issues/152252

But we can put more info about what happened in the TS Server log.

[ThaJay]

Hello, thank you for posting this feedback issue.

At the moment, it seems like many TypeScript library authors only include definition files and compiled, minimized js files. Are there any processes in motion to improve this situation? Is there a change needed in the NPM packaging process or in the TypeScript compiler?

It would be amazing for the dev experience if in the common tools, it would be the default to include source code and declaration maps. All of the publicly available NPM packages are open source so it's not like we can't read the source already and library authors could still turn off this functionality if they have specific reasons to do so.

[remcohaszing]

I would love this to be the behaviour when I use Ctrl + Click. In my experience the current behaviour of Go to Definition is pretty useless. This new Go to Source Definition option fixes that. I kind of understand why this isn’t the default behaviour from reading previous comments, but I don’t see why this behaviour can’t be opt-in.

[mahnunchik]

⌘-click => Go To Source ❤️

[RyanCavanaugh]

At the moment, it seems like many TypeScript library authors only include definition files and compiled, minimized js files. Are there any processes in motion to improve this situation? Is there a change needed in the NPM packaging process or in the TypeScript compiler?

This is already possible via declaration maps, assuming the library authors correctly configure their minimizers to also emit source maps.