Add Golang function as a source of migration

[longit644]

This PR introduces a new source of migration that enables the use of Golang functions as a source for complex logic migrations. This feature addresses the issue mentioned in #15.

Here is sample code as an example of Golang function source usage:

package main
import (
	"errors"
	"log"

	"github.com/golang-migrate/migrate/v4"
	"github.com/golang-migrate/migrate/v4/source"
	"github.com/golang-migrate/migrate/v4/source/fn"
)
func main() {
	migrations := map[string]*fn.Migration{
		"1_test": {
			Up: source.ExecutorFunc(func(i interface{}) error {
				return nil
			}),
			Down: source.ExecutorFunc(func(i interface{}) error {
				return nil
			}),
		},
	}
	d, err := fn.WithInstance(migrations)
	if err != nil {
		log.Fatalln(err)
	}
	m, err := migrate.NewWithSourceInstance("func", d, "database://foobar")
	if err != nil {
		log.Fatalln(err)
	}
	if err := m.Up(); errors.Is(err, migrate.ErrNoChange) {
		log.Println(err)
	} else if err != nil {
		log.Fatalln(err)
	}
}

[coveralls]

coverage: 58.277% (-0.2%) from 58.526% when pulling 5b15dc90e2223010b79ea41a493e60ae1c4b3aaf on fossil-engineering:feat/go-func into 23d8d33f5743c7a9f7bfb3f7bc2043f7e9348e7c on golang-migrate:master.

[longit644]

Hi @dhui, could you please review my approach and let me know if there were any limitations with my changes? Thank you for your time.

[jspdown]

@dhui I would love to help on this.

From the discussion on https://github.com/golang-migrate/migrate/issues/15 couple of strategies have been considered:

• Implemented using the golang-migrate package (this PR)
• Plugin using https://golang.org/pkg/plugin/
• Plugin using Yaegi

https://golang.org/pkg/plugin/ has several limitations that impacts negatively the user experience. You have to build the plugin with exactly the same version of the packages used by the plugin executor (same go compiler version, same arch). Therefore, updating golang-migrate would most likely forces to re-compile migration plugins.

An alternative to go plugins is the use of Yaegi (disclamer, the company I work for maintains the project). It doesn't suffer from the limitation of go plugins but comes with its own limitations. Out of the box, the biggest concern would be the lack of support for the "C" package which I suppose is used by the different database drivers. To avoid this we would have to rely on interpreter's Use to pass the compiled driver instead of interpreting it (see more on https://github.com/traefik/yaegi/issues/301#issuecomment-516772862)

Finally, the approach taken by this PR, simple and straightforward. This only drawback of this strategy is that it forces the people to rely on golang-migrate package and can no longer use the CLI tool.

If forcing the use of the golang-migrate package is not an issue for you I would definitely be in favor of what's done in this PR.

@longit644 Thanks for your work on this PR. From an API standpoint, I'm a bit worried of the Executor parameter i being an empty interface. It pushes the responsibility of knowing the underlying type to the migration implementer which is always difficult to document and use. It also prevent the compiler from catching bugs if at some point the contract changes. And finally last concern, the Exec method passes the m.db parameter which is not always enough for writing a migration. For example on MongoDB, this would prevent the use of Transactions which are held by the mongo.Client and not the mongo.Database.

No matter the strategy chosen for the migration source, there will be the same API challenges.

[longit644]

Hi @jspdown

Thank you for your insightful comment. When I started this PR, my goal was to create an idiomatic and straightforward way for handling complex migration logic that could be easily integrated with other parts of a Golang project. Therefore, I primarily focused on using this source within the project rather than with the CLI.

In my opinion, passing an empty interface is not uncommon in Golang, but it does require the developer to understand the underlying type of the migration implementer.

Regarding the Exec method, I share your concern. While it may not always provide enough context for writing migration logic, it has been sufficient in most cases I've encountered. For instance, in your MongoDB example, we can retrieve the mongo.Client from the mongo.Database. Function closures could potentially provide a workaround for this issue.

While my PR is primarily focused on usage within Golang projects, I believe that having both Golang function and Yaegi sources could be beneficial.

I appreciate your feedback and look forward to further discussion.

[casey-robertson-paypal]

Would love to see this get traction. We just found this tool and we also have the use case of Go-based migrations that we currently support in our existing tooling.

[longit644]

Hi @dhui. Please have a look at this. Thanks.

[taltcher]

Hi, @longit644 @dhui - is it something that is planned to be integrated into the package? This is a very useful feature I would like to use, I think it is a very big disadvantage that golang-migrate doesn't support golang migration files...

[longit644]

Hi @dhui , could you please take a look at this? Thank you. Do you think this feature is reasonable, or are there improvements needed to complete this PR?