Migration Script - Add rollback of locking

[Zejnilovic]

Feature

The same traversal as migration just deletes the locks.

Questions

• What if 2 datasets share a mapping table, but the mapping table should not be unlocked?

[dk1844]

Does this include rollback of migrated=true and migratedHash as well? Because since in https://github.com/AbsaOSS/enceladus/issues/2015 we are suggesting to decide on migrated flag instead of locks (because some entities may not be locked), this would get effected.

So if the point of lock clearing is to attempt to migrate again, we should clear migrated and migratedHash, too.

What if 2 datasets share a mapping table, but the mapping table should not be unlocked?

My take on this is that: yes, it should be unlocked, too, because since the point of this feature, I assume, is to relaunch the migration, that mapping table entities would get updated (new versions added) and any existing duplicates are skipped by the script anyway.

[dk1844]

Another take on (previously migrated) shared mapping: when rollbacking, take into account the migratedHash of entities specifically mentioned in the rollback, e.g. -d dataset1 - only entities matching dataset1's migratedHash will be rollbacked.

[benedeki]

The question is not just valid for mapping tables but schemas too. And the possibilities are:

• unlock all regardless other locked items depending on them
• unlock only entities that don't have other locked dependencies

Both approaches seem valid. @Zejnilovic do you have already a use-case for unlocking or just anticipating the need?

[dk1844]

@benedeki, although @Zejnilovic came with the use-case of mapping tables, I have tried to abstract and find a general approach.

The options you are suggesting are possible from locking-perpective, with @Zejnilovic we have talked about migration-session-based approach instead (technically using the migrationHash value). That way, the approach would be something in the middle - so it's basically the wider option of unlock all regardless other locked items depending on them, but with limitation of only those entities sharing migration session with the given entity (e.g. when rollbacking dataset dataset1, unlocking all related locked but only if they share the same migrationHash as dataset1 holds.

So this effectively reverts a previous migration step while unmigrating only those entities that were actually migrated in that step. (but of course, if you are reverting only part of the migration, immigration of shared dependencies may happen).

I guess we can be even stricter if we wanted to and use migration-session-based with unlock only entities that don't have other locked dependencies, too. Should we?