data-migration-tool icon indicating copy to clipboard operation
data-migration-tool copied to clipboard
verified publisher icon magento

Use multiple cores?

Open cscd98 opened this issue 7 years ago • 3 comments

The data migration tool does not appear to make use of all cores - is this possible?

I have spun up a 32 core instance, which I hoped would make this process quicker.

At the moment I am looking at a 3 hour conversion time for a Magento 1 30GB SQL database on my local machine, which is a Dell 9560 (4 core i7).

Preconditions

  1. Magento 1.14.4.0
  2. Magento 2.3.0
  3. PHP 7.2.14
  4. MySQL 5.7.25

Steps to reproduce

  1. bin/magento migrate:data ..

Expected result

  1. All cores to be used

Actual result

  1. Only 1 core used
  2. See screenshot screenshot from 2019-02-07 15-50-21

Also, what affect does bulk_size have?

cscd98 avatar Feb 07 '19 15:02 cscd98

Hi @craigcarnell

Yes, this feature would speed up migration. Internal ticket MAGETWO-72343 to implement it.

victor-v-rad avatar Feb 08 '19 09:02 victor-v-rad

@victor-v-rad Hi Victor, what is the limiting factor at the moment? It is it single CPU core speed, disk write speed, disk read speed, RAM, mysql or php/opcache optimizations?

cscd98 avatar Feb 19 '19 13:02 cscd98

This feature request was marked as "up for grab", means anyone from Magento community can take and implement.

victor-v-rad avatar Mar 15 '19 14:03 victor-v-rad