Migration utility to migrate from Glusterd1 to Glusterd2

[aravindavk]

We need a migration utility/script to start using Gd2 with existing Cluster.

Option 1:

• Install new Glusterd rpms, co exists with existing glusterd, workdir will be different and uses different port.
• Collect peer list from glusterd1 and issue peer add command and form the new glusterd2 cluster.
• In any one of the node, read the Cluster data from /var/lib/glusterd and call etcd commands to update key values directly which will be understood by glusterd2.
• Stop Glusterd1 and change the glusterd2 port to default and restart.
• Stop, disable and remove the glusterd1 services

Option 2:

• Install Glusterd2 in all nodes with different port and working dir
• In one node, Read cluster information and update etcd using etcd APIs
• Restart Glusterd2 in all nodes
• Stop Glusterd1 and change the glusterd2 port to default and restart.
• Stop, disable and remove the glusterd1 services

In both the approaches, glusterd downtime is less since all the prerequisite operations done separately without disrupting the existing Cluster. (But, don't have confidence in forming etcd cluster by updating one Glusterd2 instance as in second Option)

Please add if you have any ideas/alternatives to achieve the same.

[atinmu]

I'd vote for going with option 2 which includes both peer and volume specific information and we don't need to go through GD2's transaction model instead can write directly to etcd API which could be more less time consuming IMO.

[amarts]

Overall Cluster update when an user chooses to migrate to GD2 is like this:

• Users are allowed to perform rolling upgrade to glusterfs which is supported with GD2. (say upstream 4.1 version).

• Till all the filesystem upgrade is complete, GD1 is the default service started.

• Once filesystem is running fine, the GD1 to GD2 migration plan can kick-off.

• For both Option1 and Option2 mentioned in issue, see if we can deal with this using gdeploy like solution, so users don't get into manual errors.

[aravindavk]

I'd vote for going with option 2 which includes both peer and volume specific information and we don't need to go through GD2's transaction model instead can write directly to etcd API which could be more less time consuming IMO.

Both approach is similar wrt adding cluster data directly to etcd without going through transaction framework. Only difference is etcd cluster formation step.

[atinmu]

Isn’t “ peer add” a GD2 transaction? I was referring to that and that’s why I mentioned we need not to have GD2 transactions coming into the script.

[aweisser]

As discussed at Gluster Summit: From a users perspective it'd be great if the gd1 to gd2 update could happen without volume downtime. Keep doing a great job folks. 👍

[prashanthpai]

Isn’t “ peer add” a GD2 transaction

No it's not.

Adding directly to etcd might be the only way I can think of to add volume information. Going through REST API and txn framework will not work for obvious reasons.

There also seems to be a few things in /var/lib/glusterd w.r.t things stored by other components. Glusterd2 currently spits out volfiles with template as input. The volfile to template graph format conversion cannot be done as yet and is not worth the effort to do it.

In gd2: Graph Template + Volinfo = Volfile

To make things simpler, is it reasonable to expect (and impose) that during the upgrade to gd2, no other management operations (peer add, vol create, vol set etc) should happen ? IOW, the static information shouldn't change during migration.

[kshlm]

A heads-up to everyone.

At the Gluster Summit, I had a few discussions after my talk, where the users where concerned about the upgrade messaging I had given. After that, I had a discussion with @ShyamsundarR and @vbellur about the expected upgrade strategies. The result of these discussions is that I've tried to write up a document about the expected upgrade strategy. I've posted this to the users and devel lists for comments. Please go through the document and add your comments on the thread. We will formalize the document sometime later.

[atinmu]

@aravindavk @kshlm I see Aravinda has sent one PR https://github.com/gluster/glusterd2/pull/639 , but in yesterday's stand up Kaushal had updated some progress made on this. Request both of you to sync up on this topic to avoid duplication of efforts.

[atinmu]

As @kshlm seems to be busy on the transaction framework enhancements for GCS-Sprint1, I don't see this issue to move forward for the current sprint.

@kshlm Is there any code piece which you want to share where someone can take it forward during this sprint?