generator-jhipster icon indicating copy to clipboard operation
generator-jhipster copied to clipboard

Replace Neo4j Migrations with Liquibase for Neo4j

Open fbiville opened this issue 3 years ago • 14 comments

Signed-off-by: Florent Biville [email protected]

Fix https://github.com/jhipster/generator-jhipster/issues/16315

Please make sure the below checklist is followed for Pull Requests.

When you are still working on the PR, consider converting it to Draft (below reviewers) and adding skip-ci label, you can still see CI build result at your branch.

fbiville avatar Feb 23 '22 10:02 fbiville

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Feb 23 '22 10:02 CLAassistant

Still needs to be fully tested (work in progress).

fbiville avatar Feb 23 '22 10:02 fbiville

@fbiville Will you have time to revisit this soon?

mraible avatar Mar 24 '22 21:03 mraible

@mraible I decided to break the support into several phases. The current approach I have in mind is to just support <loadData> for Neo4j and add an explicit Liquibase change set for Neo4j that takes care of linking the loaded nodes. It will require some refactoring on Liquibase core but we got the POC for this already working on the Neo4j plugin side.

We may revisit the JHipster migration again some time later to automate the linking via <addForeignKeyConstraint>.

fbiville avatar Mar 25 '22 13:03 fbiville

@fbiville Can you please fix conflicts when you get a chance?

mraible avatar Apr 24 '22 16:04 mraible

@fbiville we have now migrated main repository to latest liquibase version (still remaining with old version for H2/mysql/mariadb due to some issues on liquibase), so you can now continue to work on this ;)

DanielFran avatar Jun 08 '22 14:06 DanielFran

@DanielFran Perfect! The changes I needed on the Liquibase core side are also released, so I believe this should finally be ready soon!

fbiville avatar Jun 08 '22 16:06 fbiville

Added bug bounty to increase motivation.

mraible avatar Aug 17 '22 08:08 mraible

Is anyone willing to continue working on this or should we close it?

mraible avatar Sep 01 '22 14:09 mraible

If we want to do this I can take a look (although I like the neo4j migrations we currently use).

atomfrede avatar Sep 01 '22 15:09 atomfrede

Had just a short call with @fbiville. We agreed to start over with this PR and target it vor v8, so we don't have a breaking change in the 7.x cycle.

atomfrede avatar Sep 16 '22 15:09 atomfrede

Great news! Thanks for the update @atomfrede.

mraible avatar Sep 16 '22 15:09 mraible

@mraible I will have a look at the neo4j + webflux issue this weekend. I should find some time.

atomfrede avatar Sep 16 '22 16:09 atomfrede

@atomfrede I'm not sure there's still an issue. I'm able to make my reactive-ms.jdl work with 7.9.3. @mshima made some changes to fix things in 7.9.2. The daily builds are failing for Neo4j though.

mraible avatar Sep 16 '22 16:09 mraible

@fbiville I've created a jhipster + neo4j + liquibase application. The result is:

 Liquibase could not start correctly, your database is NOT ready: liquibase.exception.MigrationFailedException: Migration failed for changeset config/liquibase/changelog/00000000000000_initial_schema.xml::00000000000001::jhipster:_     Reason: liquibase.exception.DatabaseException: Invalid input 'TABLE': expected "(", "allShortestPaths" or "shortestPath" (line 1, column 8 (offset: 7))_"CREATE TABLE jhi_user (id VARCHAR(255) NOT NULL, login VARCHAR(50) NOT NULL, password_hash VARCHAR(60), first_name VARCHAR(50), last_name VARCHAR(50), email VARCHAR(191), image_url VARCHAR(256), activated BOOLEAN NOT NULL, lang_key VARCHAR(10), activation_key VARCHAR(20), reset_key VARCHAR(20), created_by VARCHAR(50) NOT NULL, created_date timestamp, reset_date timestamp, last_modified_by VARCHAR(50), last_modified_date timestamp, CONSTRAINT PK_JHI_USER PRIMARY KEY (id), CONSTRAINT ux_user_login UNIQUE (login), CONSTRAINT ux_user_email UNIQUE (email))"_        ^ [Failed SQL: CREATE TABLE jhi_user (id VARCHAR(255) NOT NULL, login VARCHAR(50) NOT NULL, password_hash VARCHAR(60), first_name VARCHAR(50), last_name VARCHAR(50), email VARCHAR(191), image_url VARCHAR(256), activated BOOLEAN NOT NULL, lang_key VARCHAR(10), activation_key VARCHAR(20), reset_key VARCHAR(20), created_by VARCHAR(50) NOT NULL, created_date timestamp, reset_date timestamp, last_modified_by VARCHAR(50), last_modified_date timestamp, CONSTRAINT PK_JHI_USER PRIMARY KEY (id), CONSTRAINT ux_user_login UNIQUE (login), CONSTRAINT ux_user_email UNIQUE (email))]

liquibase.exception.LiquibaseException: liquibase.exception.MigrationFailedException: Migration failed for changeset config/liquibase/changelog/00000000000000_initial_schema.xml::00000000000001::jhipster:
     Reason: liquibase.exception.DatabaseException: Invalid input 'TABLE': expected "(", "allShortestPaths" or "shortestPath" (line 1, column 8 (offset: 7))
"CREATE TABLE jhi_user (id VARCHAR(255) NOT NULL, login VARCHAR(50) NOT NULL, password_hash VARCHAR(60), first_name VARCHAR(50), last_name VARCHAR(50), email VARCHAR(191), image_url VARCHAR(256), activated BOOLEAN NOT NULL, lang_key VARCHAR(10), activation_key VARCHAR(20), reset_key VARCHAR(20), created_by VARCHAR(50) NOT NULL, created_date timestamp, reset_date timestamp, last_modified_by VARCHAR(50), last_modified_date timestamp, CONSTRAINT PK_JHI_USER PRIMARY KEY (id), CONSTRAINT ux_user_login UNIQUE (login), CONSTRAINT ux_user_email UNIQUE (email))"

Liquibase changelogs should be reimplemented? Or this is a configuration problem?

mshima avatar Mar 24 '23 20:03 mshima

@fbiville I've created a jhipster + neo4j + liquibase application. The result is:

 Liquibase could not start correctly, your database is NOT ready: liquibase.exception.MigrationFailedException: Migration failed for changeset config/liquibase/changelog/00000000000000_initial_schema.xml::00000000000001::jhipster:_     Reason: liquibase.exception.DatabaseException: Invalid input 'TABLE': expected "(", "allShortestPaths" or "shortestPath" (line 1, column 8 (offset: 7))_"CREATE TABLE jhi_user (id VARCHAR(255) NOT NULL, login VARCHAR(50) NOT NULL, password_hash VARCHAR(60), first_name VARCHAR(50), last_name VARCHAR(50), email VARCHAR(191), image_url VARCHAR(256), activated BOOLEAN NOT NULL, lang_key VARCHAR(10), activation_key VARCHAR(20), reset_key VARCHAR(20), created_by VARCHAR(50) NOT NULL, created_date timestamp, reset_date timestamp, last_modified_by VARCHAR(50), last_modified_date timestamp, CONSTRAINT PK_JHI_USER PRIMARY KEY (id), CONSTRAINT ux_user_login UNIQUE (login), CONSTRAINT ux_user_email UNIQUE (email))"_        ^ [Failed SQL: CREATE TABLE jhi_user (id VARCHAR(255) NOT NULL, login VARCHAR(50) NOT NULL, password_hash VARCHAR(60), first_name VARCHAR(50), last_name VARCHAR(50), email VARCHAR(191), image_url VARCHAR(256), activated BOOLEAN NOT NULL, lang_key VARCHAR(10), activation_key VARCHAR(20), reset_key VARCHAR(20), created_by VARCHAR(50) NOT NULL, created_date timestamp, reset_date timestamp, last_modified_by VARCHAR(50), last_modified_date timestamp, CONSTRAINT PK_JHI_USER PRIMARY KEY (id), CONSTRAINT ux_user_login UNIQUE (login), CONSTRAINT ux_user_email UNIQUE (email))]

liquibase.exception.LiquibaseException: liquibase.exception.MigrationFailedException: Migration failed for changeset config/liquibase/changelog/00000000000000_initial_schema.xml::00000000000001::jhipster:
     Reason: liquibase.exception.DatabaseException: Invalid input 'TABLE': expected "(", "allShortestPaths" or "shortestPath" (line 1, column 8 (offset: 7))
"CREATE TABLE jhi_user (id VARCHAR(255) NOT NULL, login VARCHAR(50) NOT NULL, password_hash VARCHAR(60), first_name VARCHAR(50), last_name VARCHAR(50), email VARCHAR(191), image_url VARCHAR(256), activated BOOLEAN NOT NULL, lang_key VARCHAR(10), activation_key VARCHAR(20), reset_key VARCHAR(20), created_by VARCHAR(50) NOT NULL, created_date timestamp, reset_date timestamp, last_modified_by VARCHAR(50), last_modified_date timestamp, CONSTRAINT PK_JHI_USER PRIMARY KEY (id), CONSTRAINT ux_user_login UNIQUE (login), CONSTRAINT ux_user_email UNIQUE (email))"

Liquibase changelogs should be reimplemented? Or this is a configuration problem?

Hello @mshima, this is a configuration issue. There should not be any SQL generated (as the JDBC driver for Neo4j won't understand it, it only understands Cypher).

fbiville avatar Mar 28 '23 10:03 fbiville

@mshima has worked on this generated app FYI: https://github.com/mshima/neo4j-liquibase After a few fixes, it seems to be in a working state.

fbiville avatar Apr 12 '23 09:04 fbiville

PR is ready https://github.com/jhipster/generator-jhipster/pull/21791.

mshima avatar Apr 12 '23 11:04 mshima

@atomfrede can I go ahead and close this?

fbiville avatar Apr 12 '23 12:04 fbiville

Superseded by https://github.com/jhipster/generator-jhipster/pull/21791

DanielFran avatar Apr 12 '23 13:04 DanielFran

@DanielFran bounty claimed https://opencollective.com/generator-jhipster/expenses/140216.

mshima avatar May 22 '23 20:05 mshima

@mshima approved

DanielFran avatar May 23 '23 13:05 DanielFran