Add functionality to migrate pool data
Our initial assessment is that it could be very hard to migrate data in place. The biggest roadblock for this is that it would require making sure that there is enough space for the new metadata that would be required across v1 and v2 and data shift. This could be addressed by actually shifting the data in place or using a linear device to reorder data but both of these are error prone and add a lot of additional complexity.
We would like to consider starting with migrating data from one set of devices to another. Our current goal is to create a command that takes the same parameters as the pool create call and then the UUID of the pool to migrate the data from and perform the copy operation in kernel space temporarily using a RAID 1 array. The layer we would choose to copy would be at the layer right below the thin pool so all of the thin pool information would be preserved and everything below the thin pool would be newly created by this command.
This would require a new set of devices to migrate to, but this seems to be worth the tradeoff in terms of potential data loss with operations like moving data in place and conceptual simplicity. In place data shift would likely require at least double the storage regardless to provide enough room for a full back up in case rollback is needed in addition to the potential requirement of adding more space to the pool to accommodate the new metadata.
From the wider community, we're open to your feedback on what you would like to see in terms of data migration capabilities.