Provide transparent semless migration algorithm and migration policy

[andrii0lomakin]

1. Provide storage level seamless transparency for users' migration mechanics and implement migrations for deprecated data structures.
2. Provide a clear and understandable policy for our users that will explain our binary compatibility policy.

[tglman]

Hi,

The binary compatibility policy in OrientDB has been so far:

Main Database

• Major version can open and operate databases created by at least last minor of the previous major version
• Minor version can open and operate all the database created by all the minor of the current major version
• Hotfix do guarantee no disk compatibility breakage in some extreme cases may introduce new persistent data format but not remove old, preferably only of data structure that are resettable, like a journal.
• Disk compatibility is forward so a next version can operate database of previous version, but is not required from previous version to be able operate database of next version(it may happen if no structure change happen across versions).

Backups

Backups follow same rules of the main datastructure

Export/Import

Export/import has been guaranteed to work so far for forward( export of old version is imported in new version) and with some limitation also backward(export of new version is imported in old version) compatiblity.

Migrations of underlying structures

This can be achieved in few ways:

• No migration, new version support old structures, and keep the old structures
• Lazy migration, the structure is migrated on write, so old structure is used for read a new structure for write
• First open migration, the structure is migrated at first boot to a new structure, this kind is not allowed in hotfix release because it cause a longer downtime on update, also not suggested for minor, and major
• Export Import migration, using the database export import to reset the underlying structures -(Never Done So far even though thought ) Upgrade Database Migration, create a command that run on the database run the database migration to the newest structures(to tray to limit the outage a the minimum possible ).

This is what we have done so far on this topic, and actually I'm not looking to change much on this compatibility logic, but any feedback or evolution is welcome.

[andrii0lomakin]

@tglman Export/import is a pain that requires a lot of time, CPU, and memory resources and can not be done just by replacing the jars. And also, it always needed to be fixed. We need a user-friendly approach. So, users will replace jars, and the database will do the rest automatically.

[andrii0lomakin]

BTW, this import/export should be revised from the point of contract to implementation.