[Improvement]: Import log - add more information

[danielczestki]

Improvement description

I have XLS file with 10 rows and 2 columns. Have DataHub importer definition with mapping that columns to use Load DataObject by ID but for different classes. If the related object does not exist we see in log information like this

Could not load data object from <value> where is object ID value.

This log is useless becouse we do not have information for what column (what class) and which row this WARN occured.

Can we add this information to LOG ?

[fashxp]

sure, could you provide a PR?

[aweichler]

I would like to make these changes, but first I need to know how to realize it. To get the necessary column, row informations i need another column "data_information" (as TEXT) in the queue table. It is up to the interpreter to put the row/column information in the new column. I think the column should store a JSON object with "column" and "row" key. In the catch block of method "ImportProcessingService::processElement" it will be checked if there is log data and if so it will be appended to the message. @fashxp what do you think? I am not really sure how to create the new column, because there are no migrations in the bundle. Maybe there is an other solution for this.

[danielczestki]

I have done it already but wait for fixing this issue https://github.com/pimcore/data-importer/issues/118 and also need to change my code if this change will be applied https://github.com/pimcore/data-importer/pull/117

[danielczestki]

I think we should normalize the log. @shamoh19 solution is good but can be hard to use. If We have many columns to import, the log will have concatenate many values and it will be still hard to find row and col.

The log should be created like this (for example): Row: 10, Col: 7 - Import done + {value} Row: 12, Col: 14 - Error importing + {value}

We should add new column as @aweichler wrote to save the data_information as JSON for example where we will have { row: 5 col: 12 }

and we can add some other thing later. We can use those row and col data into log.

We need to

• create migration for queue table
• fixing the queue filler
• fixing log generate

What do You think ?

@vmalyk ?