sql-migrate icon indicating copy to clipboard operation
sql-migrate copied to clipboard

Lazy open/parse of sql files

Open KeiichiHirobe opened this issue 3 years ago • 3 comments

This is not a bug report, but a feature request. Current implementation always parse all of sql files, so may require a lot of time and memory if there are too many files or too big files in directory. But, in most of cases, we only need to parse 1~3 files because sort byId does not need content of files, it needs only the name of files. So, I think it is better to parse sql files only when we need content of files.

It seems to me that the best way to achieve this would be as follows

type Migration struct {
	Id   string
	// private field from here
	up   []string
	down []string

	disableTransactionUp   bool
	disableTransactionDown bool
	path                   string
	parsed                 bool
}

func (m *Migration) Up() []string {
	if !m.parsed {
		// call sqlparse.ParseMigration(r) and save results.
		m.parsed = true
	}
	return m.up
}

If it's okay, I can send PR.

KeiichiHirobe avatar Jan 12 '22 14:01 KeiichiHirobe

@KeiichiHirobe hey yes that's a good suggestion, feel free to submit a PR!

rubenv avatar Jan 12 '22 14:01 rubenv

@rubenv Thanks for the quick reply! I'll send PR in 1-2 weeks.

KeiichiHirobe avatar Jan 12 '22 15:01 KeiichiHirobe

@rubenv Thank you for waiting, I have just submitted PR. Please review https://github.com/rubenv/sql-migrate/pull/204.

KeiichiHirobe avatar Jan 23 '22 16:01 KeiichiHirobe