[WIP] [PARKED_FOR_NOW] Add support for MySQL 5.7 back in

[onobc]

trafficstars

ℹ️ ℹ️ This is only marked WIP to prevent merge as we are waiting on 👍🏻 for Flyway Teams license. That being said, these changes are harmless if they happen to go in before that.

This proposal adds back support for MySQL 5.7.

Overview

The previous MySQL schemas that were moved to "mariadb" have been re-instanted under "mysql57" and updated accordingly (namely sequence -> sequence table as MySQL does not support sequence).

Previous commits that removed MySQL 5.7:

• https://github.com/spring-cloud/spring-cloud-dataflow/commit/f9854a4a56aa332a4bd7323c0510e262a7271101
• https://github.com/spring-cloud/spring-cloud-dataflow/commit/a7087722e41c37705f621b49c64a39329e9ed738

Additionally the following properties need to be set on the PRO side:

spring:
  flyway:
    license-key: <redacted>
    locations: classpath:org/springframework/cloud/dataflow/server/db/migration/mysql57

Additionally the following maven pom changes need to go into PRO side:

<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-dataflow-server</artifactId>
	<exclusions>
		<exclusion>
			<groupId>org.flywaydb</groupId>
			<artifactId>flyway-core</artifactId>
		</exclusion>
	</exclusions>
</dependency>
<dependency>
	<groupId>org.flywaydb.enterprise</groupId>
	<artifactId>flyway-core</artifactId>
	<version>8.0.5</version>
</dependency>

It is not ideal that we have to duplicate the schemas, but that is what we will be doing when we support MySQL 8+ and MariaDB 10+ anyways as they are divergent enough at that point. We can look into pulling the common schemas to a shared location for them (possibly?) as only the V1 schema is currently divergent.

ℹ️ This does not update any of the k8s files under src/kubernetes as we expect only PRO customers to get MySQL 5.7 support. Is there somewhere else that does need to be updated?

Skipper

• [ ] Skipper side PR coming up w/ similar changes.

PRO

• [ ] PRO side PR coming up w/ the needed props/poms changes.

Fixes #4879

[cppwfs]

LGTM so far. It is a good questions if we can have a common schema (i.e. table creation).

[onobc]

Slack thread

We will park my PR as it does give us a “card in our back pocket” to use if the Boot 2.7 does not go well (either run into other issues or they do not fix the pieces we need).

One caveat w/ my solution is that it would require conditional env var setting in the tile and we would like to avoid that if possible.

Decision: We will ship Boot 2.7.x w/ SCDF 2.10.x as this will avoid the above caveat w/ my solution.

[onobc]

I have verified w/ the Boot 2.7.0-RC1 upgrade that both MySQL and MariaDB work well.

Have to use MySQL driver (even for MySQL 5.7)

<dependency>
  <groupId>mysql</groupId>
  <artifactId>mysql-connector-java</artifactId>
</dependency>

WHY? When using MariaDB driver it complains w/

Caused by: java.lang.RuntimeException: Driver org.mariadb.jdbc.Driver claims to not accept jdbcUrl, jdbc:mysql://localhost:3306/dataflow?autoReconnect=true&useSSL=false

We can get around this by setting ?permitMysqlScheme on url. However, at runtime it still gets reported as Maria (not MySQL) and Flyway looks in MariaDB folder and gets schema w/ sequences etc..

MySQL 5.7 Required Changes

• Add Flyway Teams
• Add MySQL Driver

[onobc]

SCDF for MySQL using MySQL driver

java -jar spring-cloud-dataflow-server/target/spring-cloud-dataflow-server-2.10.0-SNAPSHOT.jar \
  --spring.datasource.url='jdbc:mysql://localhost:3306/dataflow?&autoReconnect=true&useSSL=false' \
  --spring.datasource.username=spring \
  --spring.datasource.password=spring \
  --spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver \
  --spring.flyway.licenseKey="<<redacted>>"

[onobc]

Flyways Teams for all

The Flyway folks seemed to think its possible to use the same Flyway lib (teams) for both OSS and PRO which would greatly simplify the situation. I am waiting to hear back from there on this.

[onobc]

SCDF for MySQL using MariaDB driver

I figured out how to use the MariaDB driver - we simply need to set the spring.flyway.locations property manually as the {vendor} placeholder is derived from DB driver which is MariaDB and then the scripts are picked up for MariaDB (not MySQL).

java -jar spring-cloud-dataflow-server/target/spring-cloud-dataflow-server-2.10.0-SNAPSHOT.jar \
  --spring.datasource.url='jdbc:mysql://localhost:3306/dataflow?&autoReconnect=true&useSSL=false&permitMysqlScheme' \
  --spring.datasource.username=spring \
  --spring.datasource.password=spring \
  --spring.datasource.driver-class-name=org.mariadb.jdbc.Driver 

application.yml

spring:
  flyway:
    locations: classpath:org/springframework/cloud/dataflow/server/db/migration/mysql

[corneil]

@onobc Do you need to close this without merging?

[onobc]

Yep @corneil , I thought I did that yesterday but looks like I missed this one.

[onobc]

Superseded by #5071