allow config files (gatsby-node, gatsby-config, etc.) to be .ts files

[d4rekanguok]

• [ ] allow config to be .ts
• [ ] allow /plugins/ files to be .ts
• [ ] auto reload gatsby develop on gatsby-node.ts change

[d4rekanguok]

I've just released gatsby-plugin-node-reload, which automatically restart Gatsby on gatsby-* changes. The exit method is a bit crud (manual), but it's a stepping stone to reach auto reload with gatsby-node.ts.

[Js-Brecht]

Hey, I just saw this issue, and the brief conversation in #12. I'm curious to hear your thoughts on how you would accomplish this using a plugin. Would you use a wrapper around gatsby develop, so you can add the typescript interpreter? I always thought that it would need to be functionality included internally in Gatsby, since the interpreter would have to be spun up before reading gatsby-config and the others. TBH, I've thought about submitting a PR for that, but I get the feeling that the Gatsby team isn't very interested in the functionality.

Regarding the interpreter itself: Babel would be a definite possibility, and might be the most performant in Gatsby's context. However, Babel kind of sucks for transpiling Typescript... always wind up trying to fix some stupid issue or another, so it might become a maintenance nightmare if more people start using it. Tsc is always the best for transpiling Typescript. Spinning up a language server via the tsc api would probably be the fastest, and most robust, route; kind of like ts-jest does it. But it still might be shot down because of the time cost, just for interpreting a handful of files.

You mentioned transpiling the ts config files pre-init. I have to say, I am not a fan of that process. I did it that way for a little bit, and, well, it works... but I guess I just don't like having my source files duplicated, and dumped into the root. I even tried having it dump the transpiled config files into a separate sub directory, to keep it a bit cleaner, but that doesn't work out very well if you import files from outside of your build directory. For example, there are some times I need to import a component from src in gatsby-browser or gatsby-ssr... that just causes the tree to be duplicated in the outDir.

Ultimately, I wound up just using ts-node, with simple wrapper to start it up and configure it with any transformers I want. My .js config files are basically just two kinds of stubs:

1. Point directly to the .ts file, in the case of gatsby-browser and gatsby-ssr, since they are already being transpiled by Webpack. This is mainly just for consistency.
2. Point to my ts-node wrapper, which directs to the necessary .ts config file.

That final method is here if you're interested in checking it out.

[Js-Brecht]

I actually just stumbled on this issue, so I should amend what I said a little bit. Apparently, the Gatsby team is considering including Typescript support OOTB, including for configs

[d4rekanguok]

Hi @Js-Brecht, thank you for sharing your config & experience!! I was thinking of running tsc preinit, but I'll rethink that after reading this. I've run into the limitation that you've mentioned here as well. It looks like ts-node is a good solution.

I'm excited about TS support from Gatsby — the issue has been around for a while, but with little movement. If there's something we can do from the plugin side, even just for a few months, it'd be improvements for users.

cc/ @ricokahler

[Js-Brecht]

If there's something we can do from the plugin side, even just for a few months, it'd be improvements for users.

I agree 100%.

---

I had actually started writing a reply asking if you had any ideas how to get this to work as a plugin when I had an idea of my own. Check out this branch of my previously mentioned repository. Specifically, look at this folder, this file, and of course, the .gatsby dir.

This basically takes advantage of the Gatsby themes process, which lets you use gatsby-config (and pass it options) in a plugin. This means I can tell the plugin where to find the source files.

I created a utility hosted by the "plugin" that will generate the necessary plugin array for the root gatsby-config. All this does is make Gatsby call the gatsby-config in the plugin's directory, which sets up ts-node so that it will interpret any TS configuration files in the configured configDir, and then reads in and returns the user's TS gatsby-config. Then the plugin's gatsby-node is just a wrapper around the TS version of gatsby-node in the configDir.

I haven't figured out a way to do get it to read gatsby-browser or gatsby-ssr very easily yet, because of how Gatsby runs those files. When Gatsby runs them, all it does is a basic require on them, and then hits each of the individual api endpoints within them. Which means that all of the api endpoints need to be named exports (which we already knew). The trick is telling the plugin's modules where to find the user's TS versions. Gatsby does pass the original plugin options to those API endpoints (as the second parameter), so one way might be to hardcode all of the API endpoints Gatsby's expecting to find, and then in each of them, use the options parameter to import the user's, and pass the arguments on to the same endpoint in that one. But that sounds like it wouldn't be stable.

Thought I'd see if you had any ideas.

[d4rekanguok]

That is awesome!! My original idea was to build a wrapper around gatsby-cli & then run tsc on gatsby-node on each invocation, since I'd need the wrapper for the reload feature anyway — however @ricokahler convinced me to just get this gatsby-node.ts first, and his idea was similar to yours: requiring user's gatsby-node.ts via plugin's gatsby-node. This approach seems to be cleaner.

IIRC there were some limitation around root gatsby-node vs. plugin's gatsby-node; but that was a long time ago — perhaps that has changed now?

I haven't figured out a way to do get it to read gatsby-browser or gatsby-ssr very easily yet, because of how Gatsby runs those files.

Gatsby run those file through webpack, so they should already work as typescript files, no?

[Js-Brecht]

They do run through Webpack, so yeah, they work just fine as Typescript files. That’s why I’ve just left them in the root of the repo. However, I was trying to find a way to get ALL of that stuff out of the root, except the bare minimum (gatsby-config)

IIRC there were some limitation around root gatsby-node vs. plugin's gatsby-node; but that was a long time ago — perhaps that has changed now?

I’m not aware of any. I am pretty sure that the end-user’s api hooks take precedence over any of the plugin’s, but I’m pretty sure that plugins have a kind of cascading precedence... So the ones higher in the chain should take precedence over any sub-plugins. So I think it should work the same this way as you would expect it to, since that plugin is the first (and only) one called by the user, so it’s the highest in the chain.

I’m not 100% sure about that, though. I’d have to dig through Gatsby’s code some more to make sure.

[d4rekanguok]

That’s why I’ve just left them in the root of the repo. However, I was trying to find a way to get ALL of that stuff out of the root, except the bare minimum (gatsby-config)

Personally I prefer to leave the gatsby-*.ts files in the root, since that's Gatsby convention — but saving them in a .gatsby directory is not bad either.

Re:limitation around root gatsby-node vs. plugin's gatsby node, I think it might have changed since the introduction of theme. I remember running into issue trying to modify data that wasn't created by my own plugin.

The trick is telling the plugin's modules where to find the user's TS versions.

In a plugin we can find out the root directory via store.getState().program.directory :-/ have you tried this?

[Js-Brecht]

I remember running into issue trying to modify data that wasn't created by my own plugin.

I’m not entirely sure. Technically speaking, though, everything written in those TS files will belong to the plugin, including the plugin array. So, all of the process in gatsby-node is still happening at the same level that owns all of the plugins.

In a plugin we can find out the root directory via store.getState().program.directory :-/ have you tried this?

That’s only a problem when trying to proxy gatsby-browser and gatsby-ssr through the plugin. With gatsby-node, it’s still running in the same global scope as gatsby-config, so I just created a namespace in global with a unique symbol. That way, I was able to pass around the arguments that were provided to gatsby-config.

I guess you could use the store to get the project root; I imagine that would work as expected when the files are in the root, and not in a sub directory. Unless they’re in a “well-known” sub directory... hardcoded, in other words 😆. It’s just a matter of what convention to use; I went the path of leaving it in the user’s hands to say where their files are. The “projectRoot” and “configDir” already default to process.cwd(), so if the user doesn’t define those arguments, it will already know where the project root is (and use the Gatsby convention for config files) without accessing the store.

I get what you’re saying, about following the Gatsby convention. With the way it's set up currently, though, you’ll wind up with some issues, because you’ll have a .js gatsby-config, and a .ts. The plugin’s gatsby-config file could be hardcoded to read the .ts, but it doesn’t currently look for an extension, so it would wind up reading the same .js, and end up looping. And if you put gatsby-config in a subdir, then it’ll be there all by its lonesome 😢

Matter of fact, it’s probably best to hardcode the .ts extension when reading the user’s TS gatsby-config, considering the defaults alone will cause the issue I just mentioned 😕.

It’s all doable, I’d say. Just with a little extra here n there. If you wanted to take what I’ve done and run with it, go right ahead. I just wanted to see if I could make it work.

[d4rekanguok]

Thanks @Js-Brecht, I will give it a try. Great work on the pnpm ts starter!

[cometkim]

You can forward any serializable info from gatsby-node to browser code via adding webapack define plugin config on onCreateWebpackConfig. This is actually gatsby doing internally. just need to be careful about naming conventions.

[Js-Brecht]

@cometkim y'know, I had thought about trying that briefly, but I was done messing with it for the night. Your comment brought me back to it, so I thought I'd try it out. And voila! It works. Now I get my wish for moving all of the config files out of the root of my project 🎉

And I found a bug when running on Windows, so thank you for bringing my attention back! 🙏

Some extensive testing would need to happen with all of it, but I have to say that I'm kind of surprised how simple the whole process actually wound up being.

[Js-Brecht]

I went ahead and packaged it up and published the beta: https://www.npmjs.com/package/gatsby-plugin-ts-config

Can iterate from there, when issues come up.

[d4rekanguok]

Nice work, congrats!

[kirill-konshin]

@Js-Brecht it works, thank you!