powerapps-packagedeployer-template icon indicating copy to clipboard operation
powerapps-packagedeployer-template copied to clipboard

Add support for multithreading and CSV data migration engine features

Open ksulikow opened this issue 3 years ago • 3 comments

Data migration engine has built in support for multithreading and CSV fields format, but it is not supported by package deployer. It can be easy implemented by adding additional config parameters and using data engine features.

ksulikow avatar Apr 30 '21 14:04 ksulikow

As an example, we can use what @ksulikow has added to the samples project for the migrator itself.

https://github.com/Capgemini/xrm-datamigration/commit/ba8ac3554084bb266ecba5673fa097c5317d4e28

tdashworth avatar May 14 '21 08:05 tdashworth

For configuration, I think we can add two optional parameters for type and threads count:

<templateconfig>
    <dataimports>
        <dataimport 
            datafolderpath="ConfigurationData/Extract"
            importconfigpath="ConfigurationData/ImportConfig.json"
            importbeforesolutions="true"
            type="json"
            threadcount="4"/>
    </dataimports>
</templateconfig>

The type property should default to json although if blank a warning message should be displayed stating:

Assuming JSON as data type for import. Please specify a type for this import using type="json" to prevent this message.

The threadcount property should default to 1 and no message needs to be shown.

tdashworth avatar May 14 '21 08:05 tdashworth

@ksulikow @ewingjm please take a look at my above comment and share any thoughts you might have. 😄

tdashworth avatar May 14 '21 08:05 tdashworth