Expose a way to inject a start script into adapter-node

[Prinzhorn]

trafficstars

Is your feature request related to a problem? Please describe.

I'm tying to migrate from Sapper to SvelteKit. I'm aware of the discussion in #334 and hooks seem to solve that.

I don't think there is currently a way to have code that runs on startup with the Node adapter. There are things that need to happen once and not for every request. In Sapper my server.js looked like this:

import sirv from 'sirv';
import express from 'express';
import compression from 'compression';
import bodyParser from 'body-parser';
import * as sapper from '@sapper/server';
import { Model } from 'objection';
import knex from './lib/knex.js';
import env from './lib/env.js';

Model.knex(knex); // <==============================================

const app = express();

app.use(compression({ threshold: 0 }));
app.use(sirv('static', { dev: env.isDev }));
app.use(bodyParser.urlencoded({ extended: false }));
app.use(sapper.middleware());

(async () => {
  await knex.migrate.latest({  // <==============================================
    directory: './src/migrations',
  });

  app.listen(env.PORT, (err) => {
    if (err) {
      console.error(err);
    }
  });
})();

The first line I've marked is something that would probably not hurt if done in every hook (setting up my Objection models with the knex instance).

The second line I've marked is running the database migration. In a serverfull environment this is a rather common occurrence. I want to run the migration right before starting the server so that I'm in a consistent state between database and application code. Even if this is scaled across multiple instances it works, since the second time the migrations run it's a NOOP. I honestly have no idea how people do that in a serverless environment with a guarantee that no code is run that expects an outdated schema resulting in possibly undefined behavior.

Describe the solution you'd like

A way to run setup code before the server starts. Maybe an async function setup() {} that is awaited before the server starts? But that can't work with every adapter.

Describe alternatives you've considered

I guess some will argue I should have a second API server separated from SvelteKit and whatnot. But I want less complexity, not more.

How important is this feature to you?

I cannot migrate to SvelteKit, unless I'm missing something.

[bjon]

Why don't not just run it before starting SvelteKit? Like node db-migration.js && node [SvelteKit]?

[Prinzhorn]

Why don't not just run it before starting SvelteKit? Like node db-migration.js && node [SvelteKit]?

But I want less complexity, not more.

I can't be the only one that needs a way to setup the environment for my application and would like a central place for that?

E.g. things like this

// I _think_ the aws-sdk can do this by itself using env variables, so it might be a bad example.
require('aws-sdk').config.update({
	accessKeyId: process.env.AWS_ACCESS_KEY_ID,
	secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
	region: 'eu-west-1'
});

or

if (process.env.NODE_ENV !== 'development') {
  // This will make the fonts folder discoverable for Chrome.
  // https://github.com/alixaxel/chrome-aws-lambda/blob/eddc7fbfb2d38f236ea20e2b5861736d3a22783a/source/index.js#L31
  process.env.HOME = '/tmp';
  fs.ensureDirSync('/tmp/.fonts');
  fs.copySync(path.join(__dirname, 'fonts'), '/tmp/.fonts');
}

I could put each of them in a module so importing would execute it once. But where would I import it if there is no central place? I guess I could abuse the hooks.js for that (outside of the actual hook functions), which works for setup code that is synchronous.

I also ran into a case where a module had to be the very first thing that I required (https://github.com/lovell/sharp/issues/860#issuecomment-311611858) but SvelteKit does not give me that type of flexibility.

Another idea

So what if the node-adapter could have a flag that would make index.js not polka() on it's own, but it exports a single function that I can use? That way I can import index.js, do all the things I want and then start the server.

[Prinzhorn]

So what if the node-adapter could have a flag that would make index.js not polka() on it's own, but it exports a single function that I can use? That way I can import index.js, do all the things I want and then start the server.

Uh, maybe I'll just lazily ~require('index.js')~ await import('./index.js') in my own entry script after setup is done :tada:

[Prinzhorn]

Another real-world example: fetching metadata once during startup before launching the server https://docs.digitalocean.com/products/droplets/how-to/provide-user-data/

I'll leave this open to get some more feedback

[mankins]

So what if the node-adapter could have a flag that would make index.js not polka() on it's own, but it exports a single function that I can use? That way I can import index.js, do all the things I want and then start the server.

You could always create your own adapter, using adapter-node as a guide.

I wanted to remove the console.log on server start (and add pino), so made a quick express based adapter that's easy to modify. I agree, it would be nice to have a convention for doing things to the server startup.

[Conduitry]

Does top-level code in hooks.js get run? I would guess it probably would. You could put stuff there that you want to run once on server init and not per request.

[Prinzhorn]

You could always create your own adapter, using adapter-node as a guide.

Thanks for the hint, that sounds reasonable. I'll look into your Express adapter when I start working on this again.

You could put stuff there that you want to run once on server init and not per request.

I'd prefer not to introduce race-conditions and wait for the initialization to succeed before starting the server. I might even need some of the initialization logic to decide on certain parameters for the server. E.g. querying the hostname via cloud metadata service to configure CORS middleware. There are probably countless more use-cases.

[aewing]

I brought this up back in December, and I still think when we load the adapter in kit we should look for and await an init function, then the adapter can use that however it wants to (if at all). In the case of the node adapter, it could execute an async method passed to the adapter from svelte.config.js and we could bootstrap our pools and whatnot there without making things weird for other adapters.

I think it would be cool if the init function could pass back an initial context state for each request.

[aewing]

I don't need this, but I still want it.

[ValeriaVG]

I use a hack-around for my SvelteKit TypeScript Node project:

// Create a promise, therefore start execution
const setup = doSomeAsyncSetup().catch(err=>{
  console.error(err)
  // Exit the app if setup has failed
  process.exit(-1)
})

/** @type {import('@sveltejs/kit').Handle} */
export async function handle({ request, resolve }) {
   // Ensure that the promise is resolved before the first request
   // It'll stay resolved for the time being
    await setup;
    const response = await resolve(request);
    return response
}

Here's how it works:

• App loads hooks and starts executing the promise
• Until setup is complete all the requests are waiting
• If the setup fails - the app exits with an error
• Once setup is complete every request just jumps over await setup as it has already been resolved

Of course, having a setup hook would be cleaner and more straightforward than this. 👍

[dummdidumm]

Related to #1538

[livehtml]

@ValeriaVG big thanks for your suggestion! Can you please also show example for doSomeAsyncSetup function?

[ValeriaVG]

@livehtml Sure

const doSomeAsyncSetup = async ()=>{
   await connectToDB()
   await runMigrations()
   await chantASpell()
   await doWhateverElseYouNeedAsyncSetupFor()
}


// Create a promise, therefore start execution
const setup = doSomeAsyncSetup().catch(err=>{
  console.error(err)
  // Exit the app if setup has failed
  process.exit(-1)
})

/** @type {import('@sveltejs/kit').Handle} */
export async function handle({ request, resolve }) {
   // Ensure that the promise is resolved before the first request
   // It'll stay resolved for the time being
    await setup;
    const response = await resolve(request);
    return response
}

[parischap]

Thanks ValeriaVG for this clever code which has been very useful.

As pointed out in another thread, one issue with this solution is that the hook.js/ts file only gets loaded on first client request. And also server initialization shouldn't be in the handle hook.

So relying on your proposal and mixing it with mankins proposal to create a specific adapter, we could do the following:

Create an initialize.js/ts file that handles your initializations:

import { browser} from '$app/env';

const initializeAppCode = async () => {
  console.log('Client and server initialization');
  // Some global initializations here
  if (browser) {
    console.log('Client initialization');
    // Some client-side initializations here
  } else {
    console.log('Server initialization');
    // Some server-side initializations here
  }
};

// Start initializations as soon as this module is imported.  However, vite only loads
// the hook.js/ts module when needed, i.e. on first client request. So import this module from our own server in production mode
const initializeApp = initializeAppCode().catch((err) => {
  console.error(err);
  // Exit the app if setup has failed
  process.exit(-1);
});

export {initializeApp};

Then create a server.js/ts file in your src directory that calls your initializations:

import { path, host, port } from './env.js';
import { assetsMiddleware, kitMiddleware, prerenderedMiddleware } from './middlewares.js';
import compression from 'compression';
import polka from 'polka';
import '$lib/local_libs/initialize';  // <---- IMPORTANT LINE

const server = polka().use(
  // https://github.com/lukeed/polka/issues/173
  // @ts-ignore - nothing we can do about so just ignore it
  compression({ threshold: 0 }),
  assetsMiddleware,
  kitMiddleware,
  prerenderedMiddleware,
);

const listenOpts = { path, host, port };

server.listen(listenOpts, () => {
  console.log(`Listening on ${path ? path : host + ':' + port}`);
});

export { server };

Modify the svelte.config.js to tell node-adapter to use our server instead of the default one:

const config = {

  kit: {
    adapter: node({entryPoint: 'src/server.ts' }),  // You can pass other options if needed but entryPoint is the crucial part here
}
}

Then modify the handle hook like you proposed:

import type { Handle} from '@sveltejs/kit';
import { initializeApp } from '$lib/local_libs/initialize';

const handle: Handle = async function ({ request, resolve }) {
  // This will wait until the promise is resolved. If not resolved yet, it will block the first call to handle
  // but not subsequent calls.
  await initializeApp;

  const response = await resolve(request);
  return response;
};

There are two reasons for adding the await initializeApp; in the handle hook:

• first, it will block any request until the server is fully initialized;
• when in devmode (i.e. using npm dev), the server.ts file does not get called. So in dev mode, initialization is performed on first client request thanks to the import { initializeApp } from '$lib/local_libs/initialize'; in the hook.js/ts file.

Unfortunately, I could not come up with an elegant solution for client-side initialization. So this remains a workaround. A global init.js/ts file would still be appreciated.

[Prinzhorn]

I'm still subscribed to this issue (d'oh, it's my own). Wasn't this recently solved? You can now use entryPoint and do all the things you've always done with Express/Polka and include SvelteKit as a set of middlewares.

I haven't worked on the SveliteKit migration since opening this issue, but looking at my original example I should now be able to do literally the same thing I did in Sapper.

I can't speak for other adapters, but for adapter-node I don't see an issue any longer (in before I actually try to migrate to entryPoint and run into issues :smile: ). This is definitely much cleaner than doing any of the above workarounds using some Promise-spaghetti.

[parischap]

Take some time to read carefully my answer. In my opinion, the Promise Spaghetti (as you call it) is still needed, even after defining a new entryPoint. And, so far, we have no clean solution in dev mode and no solution at all for client-side initialization.

I'm afraid we're in for pasta for a bit longer...

[Prinzhorn]

@martinjeromelouis gotcha, please add syntax highlighting to your code blocks in the future, it's hard to read :+1: . I don't think I fully understand the problems you're having with hooks. Aren't hooks entirely unrelated? What I want to do is run initialization before even calling server.listen, no need for magic imports.

This works as custom entry point:

import { assetsMiddleware, prerenderedMiddleware, kitMiddleware } from '../build/middlewares.js';
import init from './init.js';
import polka from 'polka';

const app = polka();

app.use(assetsMiddleware, prerenderedMiddleware, kitMiddleware);

init().then(() => {
  app.listen(3000);
});

The only thing missing is an option in Vite that allows injecting code before starting the server. But what keeps us from monkey patching listen inside configureServer?

This works during dev (where it's not critical to me that init finishes before listening):

import adapter from '@sveltejs/adapter-node';
import init from './src/init.js';

const myPlugin = {
  name: 'init-when-listen',
  configureServer(server) {
    const listen = server.listen;

    server.listen = (port, isRestart) => {
      init();
      return listen(port, isRestart);
    };
  }
};

/** @type {import('@sveltejs/kit').Config} */
const config = {
  kit: {
    // hydrate the <div id="svelte"> element in src/app.html
    target: '#svelte',
    adapter: adapter({
      entryPoint: './src/server.js'
    }),
    vite: {
      plugins: [myPlugin]
    }
  }
};

export default config;

[parischap]

@Prinzhorn Thanks for the listen monkey patching. Of course this is no persistent solution but I tried to implement it and ran into the following issue : as svelte.config.js is a js file, I cannot import my init.ts module. But still it can be of some help to people who write plain javascript.

Regarding the import of the init module in your entryPoint (prod), your solution can be improved by ValeriaVG Promise proposal as I showed in the example code above. In your code, the call to the init function blocks the server until initialization is fully performed which can be annoying if the init function depends for instance on external apis that take some time to respond. With the Promise trick, you start the init asynchronously and just keep the first request pending. Of course, this only matters when you have a website with a lot of traffic.

Finally, all this does not look like a definitive solution. A regular init.js/ts file that would automatically be called by Sveltekit both at client and server start would be a much cleaner solution.

[Prinzhorn]

@martinjeromelouis I agree, it's a workaround and not a clean solution. I'm honestly overwhelmed by the complexity of SvelteKit, which is why I'm hesitant to adopt it at all. But that's a different story.

[andykais]

@Prinzhorn I tried your solution too. It works while svelte kit is in dev mode, which is great. However it is ignored entirely when building for adapter-node. Also as stated above, its outside the rest of the build process of sveltekit, so changes are not going to restart the. server, and preprocessing for things like typescript are out of the question.

I should mention I am not criticizing your design, just pointing out again the need for an official solution here.

[andykais]

perhaps the easiest solution would be to add an option to avoid lazy loading the hooks.js file? We could of course have a separate src/startup.js file which is loaded right away too. All this to say for most of us, we could get away with loading some data inside a module as singleton data, rather than complicating the process of a startup script which needs to pass args to the handle method

[Prinzhorn]

@andykais I think you missed the first code block, which is the custom entry point for adapter-node that uses the same init.js that is used in dev. I made the example in response to the new entryPoint option to offer another workaround. But I absolutely agree that this is not the way to go moving forward.

[Xananax]

entrypoint isn't always sufficient;

My build scripts need access to Svelte's context. For example:

• I heavily preprocess and generate a lot of metadata for Markdown (mdsvex) files that are collected with import.meta.glob.
• I generate a tag list from all those files, that needs to be cached
• I collect all product's descriptions, which are yaml'd in Markdown frontmatter, and upload the descriptions to Stripe to create new products.
• etc...

I started with a set of external typescript files, which would get compiled, then ran, but keeping Typescript configs and such in sync was a bother. Slight environment differences would eat a lot of debugging time. So instead, building on the handle thing, I have something a bit different:

// routes/init.ts
import type { RequestHandler } from '@sveltejs/kit'

let hasInitiated = false

export const get: RequestHandler = async () => {
  const body = hasInitiated ? true : await initSequence
  return { body }
}

const initSequence = (async (): Promise<boolean> => {
  /**
   * Do the init stuff here
   */
  return new Promise((ok) => setTimeout(ok, 3000, true))
})()

initSequence.then(() => (hasInitiated = true))

On first run, I do test $(curl -sf http://localhost:3000/init) = "true" || kill $(pgrep node)

It's rough, but simple and sufficient for my needs. Maybe it gives someone inspiration.

[Rich-Harris]

I'll confess I'm not totally sure what the feature request is here — between creating a custom server and doing setup work in src/hooks.js (where you can freely use top-level await, by the way — really helpful for this sort of thing) what is missing? (Yes, src/hooks.js doesn't get executed until the first request — though we could probably change that if necessary — but you could always just make a dummy request, even in the server itself.)

Would love to close this issue if possible!

[andykais]

@Rich-Harris I think to put it concisely, we want the ability to run arbitrary code before the server starts. The general reason for this is:

1. startup errors (like connecting to a database) are caught right away, rather than after your first user tries connecting to the server
2. no lazy loading on the first response. This is similar to the first point, but it deals with user experience. If I have to load a db, a bunch of image resources, connect to aws and whatever else the first time the hook is called, then the first user to hit my app is going to have an extremely slow response, maybe even a timeout. I would much rather do all my initialization before exposing my app to the public network.

For me personally, I am using sveltekit to build a cli web app. So you can imagine there are cases where a user passes some flags to the app which are invalid (heck, parsing args is a bit of a pain via hooks right now as well). So their workflow looks like: 1. run the app on the cli, 2. open their browser, 3. see an error, 4. go back to the terminal and cancel and restart the app with different params. I also have no ability to run something like a --help command in front of my server being started

[Conduitry]

Those all sound like things that can be addressed with a custom server entry point in the Node adapter that Rich linked to. You have complete control over when the server starts listening (or whether it starts listening at all).

[arctica]

Running the risk of having missed someting obvious: entryPoint seems to have been removed from adapter-node several months ago. How is one supposed to use a custom server now?

I found it quite surprising that something as simple as running some code during startup is seemingly so difficult or at least not obvious.

A straight forward solution would be to load hooks.ts during startup and maybe provide a listen hook while at it so code can run either right away at startup (top level hooks.ts scope) or after the server started listening.

[andykais]

@Conduitry apologies, I didn't fully understand what the custom entry point does. That will cover my use case. It does require creating an entrypoint outside the compilation of the server, but thats a pretty addressable problem. I just need to compile a typescript with some arg parsing for my own purposes

@arctica I suggest you also look at the custom server part of the readme that Rich linked above, it should let you run whatever custom logic you want. E.g. the following would allow you to effectively "load" hooks before starting your server

// src/hooks.ts
import DB from 'better-sqlite3'

class Context {
  db: DB
  status: 'uninitialized'| 'initialized' = 'uninitialized'

  async init() {
    this.db = new DB('sqlite.db') 
    this.status = 'initialized'
  }
}

export const context = new Context()


export async function handle({ event, resolve }) {
  console.log('handle request, context is', context.status)

  const response = await resolve(event);
  return response;
}

// entrypoint.js
import { handler } from './build/handler.js';
import express from 'express';
import {context} from './build/server/chunks/hooks-4f89ea9b.js'

const app = express();
app.use(handler);

;(async () => {
  await context.init()

  app.listen(3000, () => {
    console.log('listening on port 3000');
  });
})

[edit] I have played around with this a bit, and while it does work, it takes some getting used to. Any logic put into entrypoint.js is going to exist outside the sveltekit development workflow, and only be testable by building a production app and running that. A dedicated entrypoint that allows you to pass a context to handle would let sveltekit own this whole development process, but again, I am unblocked now based on what custom server allows, it just could stand to be more ergonomic. For instance, there is a build step to discover what the hooks.js file is renamed to (side note, why does it have a hashed filename, it isnt a browser dependency, so its not like we need cache-busting for it).

[arctica]

@andykais I have read that part of the wiki and it is obvious to me how that would solve the issue but it was not clear to me at all how to actually make adapter-node use that custom server because of the removal of the entryPoint option. It seems to me that one would have to (how?) get this file into the build process. That's the tricky part from my point of view as a completely new user of Sveltekit.

[andykais]

yeah as I mentioned your "entrypoint" exists outside of the sveltekit build process. It doesnt matter where you put it (mine just sits in the root of the project). All that matters is where the adapter node build folder is, because it needs to be imported into your entrypoint. For me it looks like this

sveltekit-project/
  src/
    hooks.ts
  build/
  entrypoint.js

I just run mine like so: node entrypoint.js. I agree that this goes against the philosophy of a number of sveltekit decisions that put ergonomics first, but its certainly usable.