flyway
flyway copied to clipboard
flyway
GraalVM native images support
This is more a feature request/question.
Is there any plan to support GraalVM native images out of the box in Flyway? At this moment different frameworks like Micronaut and Quarkus already support it but doing some "tricks and hacks". It would be good if there were support directly in the library so anyone that uses it can benefit.
Would you be able to explain more what GraalVM is and what problems it tries to solve? Also what 'Flyway support' would look like in your eyes.
GraalVM is a new polyglot JVM from Oracle based on OpenJDK that replaces the JIT compiler with another one written in Java (the GraalVM JIT compiler).
But in this case I'm talking about another subproject inside GraalVM, the native images: GraalVM can compile Java bytecode into native images to achieve faster startup and smaller footprint for your applications..
Every day more frameworks and libraries adapt their code to be able to generate native images in applications that use GraalVM native images. As I said, both Micronaut and Quarkus supports native-images for different kind of applications, including Flyway. The Spring team is also working on adding support to GraalVM native images.
There are different limitations https://www.graalvm.org/reference-manual/native-image/Limitations/, so framework and libraries need to be adapted to work with GraalVM. That can include different things to avoid the limitations mentioned. Sometimes it's a matter of initialize some classes at build-time or runtime. Others, as with our Micronaut Flyway integration is replacing some classes to "hack" how the migrations are discovered, because the default approach that Flyway uses (and that works with a regular JVM), doesn't work in a native-image.
For me, what I would expect when I think about "Flyway support for GraalVM native-images" is that when I use Flyway in any application and I try to convert it to a native-image, it just works. I shouldn't need to do anything, or anything apart maybe from configure something. I mean, as a user I should be able to convert my app to a native image and the migrations are discovered and applied when the app starts. As I mentioned, this doesn't happen and this moment and we needed to do some tricks in Micronaut to make it compatible. This puts on us (any framework maintainer) the "burden" of maintaining that. This is not a problem per-se, but different projects and frameworks need to do something like this. Having support in the library itself will make everything compatible without the need of having custom (and probably duplicated) code in those frameworks. Another problem is keeping that code updated, for example, Flyway 6.5.5 broke our integration and we need to adapt it to make it work again with GraalVM native-images. If the library includes everything needed we wouldn't worry about this anymore.
As a final comment, different libraries already include special configuration to support GraalVM, for example Netty.
Thank you for the detailed response. I'll add a task for us to investigate how much would need changing to get Flyway compatible.
Our current focus is releasing flyway V7, but once that is complete we can look into things like this.
Link to micronaut Flyway integration for reference: https://github.com/micronaut-projects/micronaut-flyway/tree/69fafb73ad9b9bb8309fbcfc5dbd76a16d6fece0/flyway/src/main/java/io/micronaut/flyway/graalvm
Looking forward for this nice feature update, We need this feature on Urgent bases. Actually we use Flyway db on our projects, & want to convert it's to Android 9, 10 & 11 using gluonfx-maven-plugin that's is using Graalvm native image. Without Flyway we are in loss... What is the current progress should it's will be solved urgent. I think the main problem yet is Flyway is not picking java baised migration files when running with Graalvm native.
I have created a sample project to test Flyway with HSQLDb on GraalVM native-image. Here: https://github.com/ctoabidmaqbool/Flyway-On-GraalVM-NativeImage-Test
When flyway try to looks for migration files it fails in GraalVM native-image. error:
WARN DbValidate - No migrations found. Are your locations set up correctly?
Complete log file: client-debug0.log
Note: I have used graalvm native-image argument --allow-incomplete-classpath for quick compilation otherwise it's looks for every single optional library too.
At least firstly Flyway should resolve migration files scaning in graalvm native-image running, so GraalVM users too can use it. I am not using any frameworks so I am in loss without native support in official library.
+1. But also tip for anyone using Spring Native for this. Add a @ResourceHint(patterns = "org/flywaydb/core/internal/version.txt") to your SpringApplication. The version.txt is what is causing the nullpointer. I guess graalvm provides a way to tell this resource hint manually as well.
Compile with native image runs fine using 8.5.7, but when launchin the compiled executable flyway can't find the files in db/migration.
WARNING org.flywaydb.core.internal.logging.slf4j.Slf4jLog warn Unable to scan location: c:\intellij\test\db\migration (unsupported protocol: resource)
maybe something similar like the protocol: war problem?
Runs fine without native image and happens after native-image build, and all other problems like version.txt are already working out of the box after running native agent
Edit: micronaut and Quarkus did some changes here for FlyWay in native mode, I guess scaning for classes is not supported in native mode, leaving only the other locations like filesystem:, but thats no option for client applications since it's a security issue :(
Possible solution: if scaner is not usable, add another location option to filename and classpath. since the files can be accessed directly from resources, why not using List of those files, iterating through them.
.locations("list/array/config:configfile_with_contents_of_all_Vx__Script.sql"
Why flyway team is not accounting this feature request into there todo list? Is it is very hard to implement or un-useful feature or what are the reasons for not implementing such a feature for a long time ago.
As now some of the java projects are totally made with grallvm native image. E.g. GluonVM now uses grallvm native images supports. When make android exe it use GrallVM native image and now flyway is no more of use...
Hi, I'm currently working on Flyway support for Spring Boot on native-image and wanted to chime in. We're currently having the same situation like Micronaut and Quarkus where Flyway support relies on a bunch of tricks to work. We really want to avoid that.
The are two parts to this puzzle: Flyway uses reflection, and reflection needs some metadata to work correctly on native-image. I've already contributed the metadata needed in the graalvm-reachability-metadata. It would still be nice to have the metadata right next to the code, this way if something changes in Flyway users will not be broken until the graalvm-reachability-metadata has been updated.
The other thing is that Flyway uses classpath enumeration to find the migration files. This doesn't work like it does on the JVM in a native-image. I've found a technique how to get (at least) resource enumeration to work in a native image, see this comment on the GraalVM issue. Maybe this could be incorporated into the Flyway codebase so that the migration files can be found?
Given that, is there any chance that GraalVM will be supported by Flyway in the foreseeable future?
I've implemented a small PoC to show that it's technically possible to get Flyway to run in a native-image by implementing a custom ResourceProvider: https://github.com/mhalbritter/flyway-native-image
Would be nice if the default resource provider could support the resource: protocol out of the box.
Native images are basically the future of Java and microservices, so making this issue a priority would be a great idea.
As spring v3 has out of the box support for graalvm and all of the other major players do, it's really important to get graal support I guess.
For us we love FlyWay, but we need the native-image support.
As it's over an year or more! Flyway is support system is deep sleeping. Lol!
Flyway is very good db migration tool and is liked by so many Spring developers. Now that Spring native is official it so bad that we won't be able to use flyway with it.
Flyway Team, It's high time to take some action on this very important feature
For reference: Flyway works with Spring Boot 3.0, because we added support for it in Boot directly. Nonetheless it would be great if Flyway itself would be native-image compatible.
Any progress or response in this side? Flyway is making new and new update with features. Why not taking some attention on this side?
@mhalbritter Any hints how we might use NativeImageResourceProviderCustomizer in out spring boot (3.1.0) app so we can configure flyway? Maybe you have an example that is easy to follow?
Hey @CriPstian, Flyway works with Spring Boot out of the box in a native-image. You don't have to do anything.
You are right @mhalbritter It was a misconfiguration on my side that was overriding the default behavior.
Yeah! It's very very good, if Flyway is working nicely in Spring Boot project. But still it's too very very bad if it's not working natively in Flyway without Spring Boot framework! LOL!
According to the first link, flyway does work with GraalNative if you use the second link:
- https://www.graalvm.org/native-image/libraries-and-frameworks/
- https://github.com/oracle/graalvm-reachability-metadata
Note that I've never tried yet. Could be interesting to get some feedback from someone who did it and manage to make things work 🙂
See also https://github.com/flyway/flyway/issues/2927#issuecomment-1226906759
