improve errors when create statement not found for a drop statement

[tropikoder]

When the create for a drop can't be found, at the moment the user gets this error:

Caused by: java.util.NoSuchElementException: Collection contains no element matching the predicate.
        at app.cash.sqldelight.gradle.squash.AnsiSqlMigrationSquasher.squish(AnsiSqlMigrationSquasher.kt:144)

Not only does that line not exist, it's impossible to know which drop was not matched with a create. This PR adds the drop statement to the exception

[hfhbd]

Nice! But could you please add a test?

[tropikoder]

Nice! But could you please add a test?

Where could i look in the code base for an example of a test that expects an exception? Thanks

[hfhbd]

You could just add a project to the integration tests folder: https://github.com/cashapp/sqldelight/tree/dc1080a72202566ef3a1281b74f2d72b59910950/sqldelight-gradle-plugin/src/test I would just copy this test https://github.com/cashapp/sqldelight/tree/dc1080a72202566ef3a1281b74f2d72b59910950/sqldelight-gradle-plugin/src/test/migration-failure After adding the test project, all you need is to call it in the Kotlin test code: https://github.com/cashapp/sqldelight/blob/752599954abd78e0db90ccbe5cb8b523ea4f6cf1/sqldelight-gradle-plugin/src/test/kotlin/app/cash/sqldelight/tests/MigrationTest.kt#L39 Feel free to ask for help :)

[hfhbd]

Hm, after testing the test project locally I don't get the error. And I think the test case is still a little buggy.

First, you need a dialect supporting DROP COLUMN. If you don't specify a dialect, sqldelight uses sqlite 3.18, which does not support drop columns. dialect("app.cash.sqldelight:sqlite-3-35-dialect:${app.cash.sqldelight.VersionKt.VERSION}") supports DROP COLUMN but it works...

AFAIK Migration squashing is useless with only one sqm file. So you need two migration files to squish them into one file. If you use migrations, you should enable deriveSchemaFromMigrations and move your CREATE statements in an initial migration file too (1.sqm) and your DROP COLUMN into 2.sqm.

If you do this, I don't get this error java.util.NoSuchElementException: Collection contains no element matching the predicate. but no error, which looks like a bug too, because it should fail when referencing a missing column...

While I understand your initial fix in the ANSI Sql-squasher, I am curious how you got this error at all.

[tropikoder]

Thanks for the help. You are correct in your suggestions. I asked a colleague who has Android Studio to run the pipeline and his machine gets an out of memory error. The PR would be better if we could actually run locally.

I suspect I got the error because we may have an index in the DB that was not created via a migration. We have hundreds of migrations (hence the desire to squash) so it's hard to know which one causes the error (hence the PR).

[tropikoder]

Also, there are some things in the tests that i've never used and I didn't find docs for, such as 1.db. I made a new commit that removes it.

[hfhbd]

@tropikoder You don't need Android Studio, I don't use it too. For debugging just link the project by right-click the gradle.build file and test it. Or simple run the test.

If you get OOM errors, the cause is often dokka and you need to increase the heap:

org.gradle.jvmargs=-Xmx2048m
org.gradle.parallel=true

I am fine with the actual fix, but I would like to have 1) a working test :) and 2) the config causing your crash, because I don't understand why the crash happened at all. AFAIK this method is always overwritten.

[tropikoder]

Is there a way to build just the jvm jars so i can run my fork locally from my main project (which has the hundreds of migrations and squash problem)? Thanks!

[hfhbd]

@tropikoder Sure, just use either publishAllPublicationsToInstallLocallyRepository (the folder will be at sqldelight/build/localMaven or use publishToMavenLocal and use repositories { mavenLocal() }. And be sure to update your version to 2.0.0-SNAPSHOT.

[tropikoder]

Thanks @hfhbd

My colleague was able to run the locally installed jars from our fork. It seems that there's a bug when migrations drop views (they get mixed up with tables). With our changes, we are able to generate exceptions like this: Create statement not found for drop table statement View_Product_ProductPriceAndStock

[tropikoder]

using the local jars we noticed that the matching of create/drop statements fails when the View drop statement has IF EXISTS

[tropikoder]

using the local jars we noticed that the matching of create/drop statements fails when the View drop statement has IF EXISTS

Actually there are mismatches even without IF EXISTS

[hfhbd]

@tropikoder

It seems that there's a bug when migrations drop views (they get mixed up with tables)

Do you have a sample? Or does the test work?

[tropikoder]

@hfhbd i couldn't make a PR that reproduces the error.

We gave up on the automated squash and did it manually.