Refactor the data migration code

[caipengbo]

Search before asking

• [X] I had searched in the issues and found no similar issues.

Motivation

Now that we have support for DBIterator and WALIterator, we can refactor migration code based on redis-command using these two classes. Also, now that we support two different migration types, we can make the code base about migration a bit cleaner than it is now.

The unit tests could also be clearer.

Solution

No response

Are you willing to submit a PR?

• [ ] I'm willing to submit a PR!

[wsehjk]

Hi, i'm interested in this issue, could you please assign this to me?

[tisonkun]

@wsehjk Assigned. Go ahead :D

Based on my experience, you can share your opinion on this issue and how you'd like to implement it. If it's trivial, directly sending a patch is good.

Barely asking for an assignment without any thoughts or progress attempts often ends with giving up.

[git-hulk]

@caipengbo Would you mind giving some input about what you plan to improve?

[caipengbo]

@git-hulk This is my personal opinion for reference only:

1. decouple the iteration logic from the send logic:
   • use DBIterator and WALIterator for data iteration
   • use a Sender helper class for sending data and rate limit
2. Make the two migration types more uniform
3. Refactor Golang unit tests about data migration to make them more concise and faster to run
4. Other minor change:
   • MigrationJob stores what is really needed
   • ...

As @tisonkun said, you can also share some of your plans and we can discuss them together @wsehjk .

[git-hulk]

@caipengbo Thanks for your input.

[wsehjk]

@git-hulk This is my personal opinion for reference only:

1. decouple the iteration logic from the send logic:

   • use DBIterator and WALIterator for data iteration
   • use a Sender helper class for sending data and rate limit

2. Make the two migration types more uniform

3. Refactor Golang unit tests about data migration to make them more concise and faster to run

4. Other minor change:

   • MigrationJob stores what is really needed
   • ...

As @tisonkun said, you can also share some of your plans and we can discuss them together @wsehjk .

Thank you for the info, really helpful, and i will walk through the code and share my thought before developing