hyades icon indicating copy to clipboard operation
hyades copied to clipboard

Generate DB migration script during release, and attach it as release artifact

Open nscuro opened this issue 8 months ago • 1 comments

Current Behavior

Database migrations are currently performed automatically during startup of the API server.

While it's simple for users and provides a hands-off experience for upgrades, it can be problematic in scenarios where migrations must be applied manually. This can be the case for regulatory, organizational, or practicality reasons. The latter might be applicable if the database has grown so large, that attempts to do "live" migrations have no chance of succeeding.

We already have a bash script to generate a SQL migration script from our Liquibase changelog: https://github.com/DependencyTrack/hyades-apiserver/blob/main/dev/scripts/dbschema-generate-migration.sh. It would be ideal if users wouldn't have to run this themselves.

Proposed Behavior

Generate the migration script as part of every release, and attach it to the GitHub release.

Extend the database operations documentation to include instructions for manual application of migrations.

Checklist

nscuro avatar Mar 27 '25 17:03 nscuro

Just dawned on me that simply generating the script from a blank database is not sufficient in all cases.

In some scenarios like #1763 we need to perform actions that cannot be done via SQL alone. With Liquibase we can work around this by implementing CustomTaskChanges or CustomSqlChanges.

  • CustomTaskChange involves Java code executing SQL commands. Naturally whatever happens here cannot magically be exported to a SQL migration script.
  • CustomSqlChangeinvolves Java generating SQL statements. These will appear in SQL migration scripts.

In both cases, the actions performed will depend on the current database content. In #1763, we are migrating existing API keys. Those will vary widely between DT instances.

Providing a simple migration script is thus not sufficient. What we need is the ability for users to execute the dbschema-generate-migration.sh script against their database.

At the moment executing this script requires the API Server source code, as well as Java and Maven to be present. This is not great. Java is the most we can reasonably ask for here.

Instead of generating a script, we need to improve our tooling to make it easier for users to generate their own.

nscuro avatar Apr 17 '25 12:04 nscuro