[FEATURE] Adds support to optionally migrate directus tables

[marcemira]

Description

• It basically enables migrations on given list of directus tables.
• Adds a new argument --allowedDirectusTables which takes a string of comma separated table names (i.e. directus_users,directus_files)

Motivation / Use Case

Directus tables are allowed to be modified, in my case, I need a Directus User from directus_users to have a one-to-many relationship with one of my collections. This works fine, but we needed to be able to migrate that change as well.

[marcemira]

hey @dstoyanoff I was just deploying a fresh copy and trying this change one last time from scratch, and noticed that if you start the whole process with something like --allowedDirectusTables=directus_users it's all good, until you run another migration but without the argument... it produces a migrations with an up like this:

drop table "directus_users"

I'm not sure why tho... but seems like once you start doing migrations with this argument, changing results in table drops? Any idea what's going on? EDIT: Also, starting without the enabled directus tables, and adding them later on, produces an up like:

create table "directus_users"

Clearly, it's something that's more suitable of being configurable, and must not change since the first migration (not great ...), thoughts?

[dstoyanoff]

hey @dstoyanoff I was just deploying a fresh copy and trying this change one last time from scratch, and noticed that if you start the whole process with something like --allowedDirectusTables=directus_users it's all good, until you run another migration but without the argument... it produces a migrations with an up like this:

drop table "directus_users"

I'm not sure why tho... but seems like once you start doing migrations with this argument, changing results in table drops? Any idea what's going on? EDIT: Also, starting without the enabled directus tables, and adding them later on, produces an up like:

create table "directus_users"

Clearly, it's something that's more suitable of being configurable, and must not change since the first migration (not great ...), thoughts?

Well, the state file keeps track of all changes being made. In this case, since the system table is ignored, it thinks it got deleted. Perhaps as an additional measurement on this PR, you could add a check to ignore drop/create of system tables. WDYT?

[marcemira]

Well, the state file keeps track of all changes being made. In this case, since the system table is ignored, it thinks it got deleted. Perhaps as an additional measurement on this PR, you could add a check to ignore drop/create of system tables. WDYT?

The problem seem to be that once we enable an allowed set of direcuts tables, removing them will create a diff that produces the DROP/CREATE. In order to prevent that, we could skip these by saving the last value of --allowedDirectusTables diff for changes, and skipping accordingly. Does that makes sense?

[dstoyanoff]

Well, the state file keeps track of all changes being made. In this case, since the system table is ignored, it thinks it got deleted. Perhaps as an additional measurement on this PR, you could add a check to ignore drop/create of system tables. WDYT?

The problem seem to be that once we enable an allowed set of direcuts tables, removing them will create a diff that produces the DROP/CREATE. In order to prevent that, we could skip these by saving the last value of --allowedDirectusTables diff for changes, and skipping accordingly. Does that makes sense?

I think we should just never DROP/CREATE a system table, which would be easier to implement and will cover more scenarios

[marcemira]

I think we should just never DROP/CREATE a system table, which would be easier to implement and will cover more scenarios

Sounds good!

[julbd]

@dstoyanoff What's the status on this ? :)

[dstoyanoff]

I believe there were some changes to be made by @marcemira. However, this package isn't well maintained since Directus launched their own migrations functionality. Any reason why you are not using it?