cli icon indicating copy to clipboard operation
cli copied to clipboard

ESM & sequelize db:migrate

Open mahnunchik opened this issue 2 years ago • 28 comments

It would be great if sequelize-cli supports migrations in ESM way.

What you are doing?

// db/migrations/20220924162319-create-test.js
export async function up(queryInterface, Sequelize) {
  await queryInterface.createTable('test', {
    id: {
      type: Sequelize.DataTypes.INTEGER,
      allowNull: false,
      autoIncrement: true,
      primaryKey: true,
    },
    createdAt: {
      allowNull: false,
      type: Sequelize.DataTypes.DATE,
    },
    updatedAt: {
      allowNull: false,
      type: Sequelize.DataTypes.DATE,
    }
  });
}
export function down(queryInterface, Sequelize) {
  return queryInterface.dropTable('test');
}
npx sequelize db:migrate

What do you expect to happen?

Migration works.

What is actually happening?

npx sequelize db:migrate

Sequelize CLI [Node: 16.16.0, CLI: 6.4.1, ORM: 6.23.0]

Loaded configuration file "db/config.js".
== 20220924162319-create-test: migrating =======

ERROR: require() of ES Module /test/db/migrations/20220924162319-create-test.js from /test/node_modules/umzug/lib/migration.js not supported.
Instead change the require of 20220924162319-create-test.js in /test/node_modules/umzug/lib/migration.js to a dynamic import() which is available in all CommonJS modules.

Output, either JSON or SQL

Dialect: postgres Database version: 12 Sequelize CLI version: 6.4.1 Sequelize version: 6.23.0

mahnunchik avatar Sep 19 '22 19:09 mahnunchik

Related issue https://github.com/sequelize/umzug/issues/475

mahnunchik avatar Sep 19 '22 19:09 mahnunchik

Amazing how we hit this issue at the exact same time yesterday!

bobber205 avatar Sep 20 '22 16:09 bobber205

Replacing: return require(this.path); with return import(this.path); solves it for ES Modules.

Monkey patch from the original issue works...

mahnunchik avatar Sep 20 '22 17:09 mahnunchik

@mahnunchik Where did you do this change set?

bobber205 avatar Sep 20 '22 17:09 bobber205

@bobber205 sequelize-cli uses pretty old 2.3.0 umzug module:

https://github.com/sequelize/umzug/blob/v2.3.0/src/migration.js#L68

mahnunchik avatar Sep 20 '22 17:09 mahnunchik

sequelize-cli uses pretty old 2.3.0 umzug module:

https://github.com/sequelize/umzug/blob/v2.3.0/src/migration.js#L68

Indeed. We will update to the latest umzug release in v7 of the CLI.

WikiRik avatar Sep 21 '22 21:09 WikiRik

As a temporary workaround to not have to rename generated migration files to .cjs, in your migrations folder, create a package.json file with contents

{
  "type": "commonjs"
}

or, since commonjs is the default, simply

{}
image This tricks Node into thinking that your migration folder is a commonjs package, and allows you to use `npx sequelize migration:create` and `npx sequelize db:migrate` the same way as you'd use it in a commonjs project.

1valdis avatar Nov 25 '22 09:11 1valdis

sequelize-cli uses pretty old 2.3.0 umzug module: https://github.com/sequelize/umzug/blob/v2.3.0/src/migration.js#L68

Indeed. We will update to the latest umzug release in v7 of the CLI.

Hey @WikiRik, can you provide an expected timeline for when v7 will be released?

taslangraham avatar Jan 03 '23 16:01 taslangraham

@taslangraham not yet, we'll start work when sequelize v7 is in beta (which we expect in the next month or so)

WikiRik avatar Jan 03 '23 16:01 WikiRik

@WikiRik thanks for the update!

taslangraham avatar Jan 03 '23 17:01 taslangraham

Does this provide a solution? https://github.com/sequelize/cli/pull/982 If so, how would an ESM .sequelizerc look?

There's also this docs change, but it doesn't mention .sequelizerc https://github.com/sequelize/sequelize/pull/13669/commits/81534441313f83bde446b2be6e88a7fa363e142d

mariusa avatar Mar 27 '23 09:03 mariusa

It's been over 9 months that this issue is going on now. Not supporting ESM is a real deal-breaker for many people as ESM is superior to CJS in many ways that are now common knowledge. If you need help migrating to a newer version of umzug I would be happy to help by opening a PR. umzug 2.3.0 is now over 3 years old, it's most likely no longer being maintained. It's becoming critical to update this dependency.

In any case, just let me know if you need a PR to move to ESM-first with a secondary CJS build. I would be happy to help if this can fast-track this problem for everyone.

SmashingQuasar avatar Jun 28 '23 08:06 SmashingQuasar

We're currently planning a full rewrite from scratch of the CLI for v7, but we're happy to review PRs to upgrade umzug and allow ESM builds if it does not have any breaking changes

WikiRik avatar Jun 28 '23 08:06 WikiRik

We're currently planning a full rewrite from scratch of the CLI for v7, but we're happy to review PRs to upgrade umzug and allow ESM builds if it does not have any breaking changes

I understand that, you mentioned it on January 3rd. The thing is, it's not tied to the CLI, so the full rewrite will not solve the issue. The problem lies within umzug itself. Even in it's current state, it is still written and built using CJS. Since umzug is using a dynamic require, a rewrite of umzug is necessary. Unfortunately, this issue is affecting more thant Sequelize. Several ORMs are using umzug and reporting the same problem. It's relatively widespread. Since Sequelize seems to also be maintaining umzug, would you be open to a PR migrating it to ESM-first?

SmashingQuasar avatar Jun 28 '23 08:06 SmashingQuasar

umzug is indeed under our organization, but it is being maintained by @mmkal Please open an issue there if changes need to be made to umzug first

WikiRik avatar Jun 28 '23 08:06 WikiRik

@WikiRik @mmkal Any news? It seems umzug has been updated.

mahnunchik avatar Oct 08 '23 05:10 mahnunchik

Umzug still uses require and is commonjs by default, but it supports import(...) via a custom resolver. So if sequelize-cli wants to do things this way it should be possible already. @WikiRik there's a demo of this in the umzug repo under the examples/ folder.

mmkal avatar Oct 08 '23 15:10 mmkal

FWIW I've done as others have suggested and renamed my migration files to end with .cjs instead of .js in my "type": "module", project and things are working well. Annoying but easy enough to workaround.

bobber205 avatar Oct 09 '23 16:10 bobber205

@bobber205 It doesn't work if migration files in ESM 😆

mahnunchik avatar Oct 09 '23 16:10 mahnunchik

@WikiRik ping

mahnunchik avatar Oct 20 '23 03:10 mahnunchik

@bobber205 It doesn't work if migration files in ESM 😆

Hey good afternon

did you resolved this issue? i have the same problema when im deploying Captura de tela 2024-01-12 153506

luiskurihara avatar Jan 12 '24 18:01 luiskurihara

Umzug now fully supports esm. This repo will need to update. It's a major version update with several breaking changes, the version this repo uses is about four years old. The umzug repo has docs on how to update to v3. @WikiRik may know more about if anyone is already working on this.

mmkal avatar Jan 12 '24 23:01 mmkal

Upgrade to umzug v3 will happen with v7 of the CLI which will also support v7 of sequelize. No ETA or roadmap yet, but we'll work on it once we release the first v7 beta

WikiRik avatar Jan 12 '24 23:01 WikiRik

@WikiRik @mmkal at least I won't be switching to sequelize v7 for a few more years. Almost all code is already in ESM format, so it would be very convenient to update the umzug library for sequelize version v6 too.

mahnunchik avatar Jan 12 '24 23:01 mahnunchik

With the community we might be able to accommodate for using both v2 and v3 of umzug in cli v6, but since umzug v3 does have its breaking changes we as maintainers will give limited attention to including it in v6. We will of course help out in making the upgrade process to v7 smoother and better documented

WikiRik avatar Jan 13 '24 00:01 WikiRik

A sample code for who, like me, prefer to have everything in ESM. This code allows you to keep migrations in ESM and not use sequelize cli at all.

// migrator.js
import path from 'node:path';
import { pathToFileURL } from 'node:url';
import sequelize from './db/index.js';
import {
  SequelizeStorage,
  Umzug,
} from 'umzug';

const umzug = new Umzug({
  migrations: {
    glob: ['db/migrations/*.js', {
      cwd: path.dirname(import.meta.url.replace('file://', '')),
    }],
    resolve(params) {
      const getModule = () => import(pathToFileURL(params.path));
      return {
        name: params.name,
        path: params.path,
        up: async upParams => (await getModule()).up(upParams),
        down: async downParams => (await getModule()).down(downParams),
      };
    },
  },
  context: { sequelize },
  storage: new SequelizeStorage({ sequelize }),
  logger: console,
});

await umzug.runAsCLI();

console.log('done');
await sequelize.close();
  "scripts": {
    "migrate": "node migrator.js up",
    "migrate:down": "node migrator.js down"
  },

mahnunchik avatar Jan 13 '24 00:01 mahnunchik

try importing babel , it worked for me

https://sequelize.org/docs/v6/other-topics/migrations/#using-babel

vickrant avatar Jan 14 '24 13:01 vickrant

@vickrant You don't need babel since node 12

mahnunchik avatar Jan 15 '24 04:01 mahnunchik