Support for sbt?

[jec]

I'm using the Neo4j Java lib in a Scala app, and Neo4j-Migrations looks like a really useful way to maintain constraints and such. However, my workflow is entirely in sbt, and I wonder how I might get this lib to work with that. Does anyone have any insight into that? Is anyone aware of a blog post somewhere that describes it?

Cheers!

[michael-simons]

Hey. Thanks for your kind comment. I’m happy to look into that. In the end, it shouldn’t be too hard to call the core api from Scala and sbt. It’s just not my main expertise.

Here’s how I do this for Maven

https://github.com/michael-simons/neo4j-migrations/blob/902b78213e5925fa1fdf14784afbad0650cc70a3/neo4j-migrations-maven-plugin/src/main/java/ac/simons/neo4j/migrations/maven/AbstractConnectedMojo.java#L117

and as an example the migrate mojo

https://github.com/michael-simons/neo4j-migrations/blob/902b78213e5925fa1fdf14784afbad0650cc70a3/neo4j-migrations-maven-plugin/src/main/java/ac/simons/neo4j/migrations/maven/MigrateMojo.java#L41

if you could point me to some resources where I can learn about

• sbt
• Extending it

Would be a good start

[jec]

Hi Michael. Thanks for your quick response. The sbt docs related to building plugins for it appear to begin here:

https://www.scala-sbt.org/1.x/docs/Best-Practices.html

I'll explore what it takes to do such a thing as well. Thanks again!

[michael-simons]

I’ve some code working in principle but it seems that sbt (or Scala) has some issues with Java service loader API that migrations uses internally. As in: no services are discovered.

See those. They work just fine in Spring, Quarkus, CLI , Maven and native image.

I don’t have the right Google foo in Scala terms to find what’s needed to make them work. Do you? The rest is afaik mainly boiler plate and translating into sbt. https://github.com/michael-simons/neo4j-migrations/blob/main/neo4j-migrations-core/src/main/resources/META-INF/services/ac.simons.neo4j.migrations.core.ResourceBasedMigrationProvider

[michael-simons]

I have something working now… It would be most awesome if we can work together on this as a starting point. Given

organization := "eu.michael-simons.neo4j"
name := "sbt-neo4j-migrate"
version := "0.1"
sbtPlugin := true

scalaVersion := "2.12.16"
javacOptions ++= Seq("--release", "8")

libraryDependencies += "eu.michael-simons.neo4j" % "neo4j-migrations" % "1.7.1"

plus

package sbthello

import ac.simons.neo4j.migrations.core.{Migrations, MigrationsConfig}
import org.neo4j.driver.*
import sbt.*
import sbt.Keys.*
import sbt.internal.inc.classpath.ClasspathUtil

import java.util.logging.Level

object HelloPlugin extends AutoPlugin {
  override def trigger = allRequirements

  override def requires = sbt.plugins.SbtPlugin

  object autoImport {

    val neo4jAddress = settingKey[String]("The jdbc url to use to connect to the database.")
    val neo4jUser = settingKey[String]("The user to use to connect to the database.")
    val neo4jPassword = settingKey[String]("The password to use to connect to the database.")

    val neo4jLocations = settingKey[String]("Locations on the classpath to scan recursively for migrations. Locations may contain both sql and code-based migrations. (default: classpath:neo4j/migrations)")

    val neo4jInfo = taskKey[Unit]("Retrieves the complete information about the migrations including applied, pending and current migrations with details and status.")
  }


  import autoImport.*

  override lazy val projectSettings: Seq[Setting[_]] = Seq(
    neo4jInfo := {
      val address = neo4jAddress.value
      val user = neo4jUser.value
      val password = neo4jPassword.value

      val config = MigrationsConfig.builder()
        .withLocationsToScan(neo4jLocations.value)
        .build()
      val migrations = withContextClassLoader((Compile / externalDependencyClasspath).value)(new Migrations(config, openConnection(address, user, password)))
      val s = streams.value
      s.log.info(migrations.info().prettyPrint())
    }
  )

  private def openConnection(address: String, user: String, password: String): Driver = {
    GraphDatabase
      .driver(
        address,
        AuthTokens.basic(user, password), createDriverConfig
      )
  }

  private def createDriverConfig = Config.builder
    .withLogging(Logging.console(Level.SEVERE))
    .withUserAgent(Migrations.getUserAgent)
    .build

  private def withContextClassLoader[T](cp: Types.Id[Keys.Classpath])(f: => T): T = {
    val classloader = ClasspathUtil.toLoader(cp.map(_.data), getClass.getClassLoader)
    val thread = Thread.currentThread
    val oldLoader = thread.getContextClassLoader
    try {
      thread.setContextClassLoader(classloader)
      f
    } finally {
      thread.setContextClassLoader(oldLoader)
    }
  }
}

I can have a 2nd project looking like this

name := "untitled4"
version := "0.1"

neo4jAddress := "bolt://localhost:7687"
neo4jUser := "neo4j"
neo4jPassword := "secret"

neo4jLocations := {
  val path = (Compile / packageBin / classDirectory).value
  new File(path, "/neo4j/migrations/").toURI.toString

}

enablePlugins(sbthello.HelloPlugin)

that will spit out

untitled4 sbt neo4jInfo
[info] welcome to sbt 1.6.2 (Oracle Corporation Java 1.8.0_211)
[info] loading global plugins from /Users/msimons/.sbt/1.0/plugins
[info] loading settings for project untitled4-build from plugins.sbt ...
[info] loading project definition from /Users/msimons/Projects/tmp/untitled4/project
[info] loading settings for project untitled4 from build.sbt ...
[info] set current project to untitled4 (in build file:/Users/msimons/Projects/tmp/untitled4/)
[info] 
[info] neo4j@localhost:7687 (Neo4j/4.4.4)
[info] Database: neo4j
[info] 
[info] +---------+-------------+--------+--------------+----+----------------+---------+---------------------------+
[info] | Version | Description | Type   | Installed on | by | Execution time | State   | Source                    |
[info] +---------+-------------+--------+--------------+----+----------------+---------+---------------------------+
[info] | 0001    | A migration | CYPHER |              |    |                | PENDING | V0001__A_migration.cypher |
[info] +---------+-------------+--------+--------------+----+----------------+---------+---------------------------+
[success] Total time: 1 s, completed 17.06.2022 23:09:50

Wdyt @jec is that something we can start with?

[jec]

@michael-simons this looks great! Very quick response, unlike mine. I would like to work with you on this as time allows. This involves a side project of mine, so it wouldn't be during my office hours (east coast USA). What do you see as the next steps?

[michael-simons]

So, I would like to discuss deployment first:

I have no clue how to deploy such thing :) Maven doesn't seem to be an option. I would like to keep things in this mono repo, though, so I would see this as an addtional folder / project in here.

That would need some scripting to sync versions and something that triggers sbt release from within maven during maven release. I could do the latter, but need help to understand sbt release.