cqlmigrate
cqlmigrate copied to clipboard
CQL migrate fails due to race condition of updating schema metadata after acquiring lock
When deploying to 2 nodes, the first gets the lock, runs migrate successfully, happy days.
The second then got the acquired the lock, skips the bootstrap but doesn't seem to have the updated schema metadata so it tries to apply the "create schema updates table".
This fails with an exception as it already exists.
[2017-01-06 11:58:09,563] lcm INFO [main] u.s.c.Lock - [] Attempting to acquire lock for 'channel_metadata.schema_migration', using client id '93155931-75b4-4fae-8be9-3dc51c111f4c' [2017-01-06 11:58:09,584] lcm INFO [main] u.s.c.CassandraLockingMechanism - [] Lock currently held by Row[false, channel_metadata.schema_migration, 948f2e5f-8447-4093-bd7f-cf32be7289f1] [2017-01-06 11:58:10,095] lcm INFO [main] u.s.c.CassandraLockingMechanism - [] Lock currently held by Row[false, channel_metadata.schema_migration, 948f2e5f-8447-4093-bd7f-cf32be7289f1] [2017-01-06 11:58:10,644] lcm INFO [main] u.s.c.CassandraLockingMechanism - [] Lock currently held by Row[false, channel_metadata.schema_migration, 948f2e5f-8447-4093-bd7f-cf32be7289f1] [2017-01-06 11:58:11,171] lcm INFO [main] u.s.c.CqlMigratorImpl - [] Loading cql files from [/tmp/cqlmigrate7688832900537474535] [2017-01-06 11:58:11,180] lcm INFO [main] u.s.c.KeyspaceBootstrapper - [] Keyspace found, not applying bootstrap.cql [2017-01-06 11:58:11,206] lcm ERROR [main] u.s.c.CqlLoader - [] Failed to execute cql statements [CREATE TABLE schema_updates (filename text primary key, checksum text, applied_on timestamp);]: Table channel_metadata.schema_updates already exists
possible fix: use CREATE IF EXISTS, and remove metadata check.
Thanks @chrishughes25 for finding this. Do you mind submitting a PR?