Import feature: show the differences of an element when it is updated by an import (and refactor)

[MyPyDavid]

Description

This PR implements a text difference viewer for the import of RDMO content elements (catalogs, optionsets, etc..). It uses the react-diff-viewer-continued to display the changes during an import. In the back-end a sort of "track changes" function tracks the current and new state of each element which is returned in the json response.
The code architecture of the import is refactored into one central import function under management with helpers for each type of element.

Related issue: #468

Motivation and Context

How has this been tested?

Screenshots (if appropriate)

Currently:

Types of Changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [x] Code style update (formatting, renaming)
  • some type hints have been added to the new code
• [x] Refactoring (no functional changes, no api changes)
• [ ] Build related changes
• [ ] Documentation content changes
• [ ] Other (please describe):

Checklist

• [x] I have read the contributor guide.
• [x] My code follows the code style of this project.
• [ ] My change requires a change to the documentation.
• [ ] I have updated the documentation accordingly.
• [x] I have added tests to cover my changes.
  • [x] also added more front end e2e tests
• [x] All new and existing tests passed.

Still todo

Front-end

• [x] add filter on top of import page
• [x] add info messages summary on top of import page
• [x] button to show only the changed elements

[MyPyDavid]

• [x] fix the actions in the sidebar

[jochenklar]

I am sorry, there is more work to do on the css. Paddings and margins are off all over the place.

[jochenklar]

The padding needs to align with the rest of the panel.

I think we should only use the <code> for URIs or Element/Model names. Just Normal font should suffice. Also the "Changed:" label in the front is redundant.

[jochenklar]

Same here around the warning. It think it 15px.

[jochenklar]

The different items need the same margin-bottom, also here the margin-left needs to be gone.

[jochenklar]

This needs more styling, the background should be white (except the red/green part), Current/plus/minus looks like they are clickable, but they are not. The whole block needs a margin-bottom of 10 or 20. The mono space font should be smaller.

[jochenklar]

This also needs more separation, and Upper case letters at the beginning. Total is probably "Gesamt" in German.

[jochenklar]

Here the indentation is off.

[MyPyDavid]

Same here around the warning. It think it 15px.

thanks yes, I had panel-body <div> at the wrong place in the hierarchy of the component

[MyPyDavid]

it looks somewhat better now I think, however there are still some todos...

[jochenklar]

Yes, much better, but there is now another panel around the entries, I would rather remove it. panel-body and list-group can be combined like this: https://getbootstrap.com/docs/3.4/components/#panels-list-group

[jochenklar]

I would also remove the red and green background (it conflicts with the border radius of all other panels buttons etc.). The plus and minus look like buttons, you could remove the border and add the light red and blue there.

We should also remove the "row/col indentation" from the displayed fields. Maybe we put the field name on top, and display either the diff or just the fields value, a bit like this:

help_de Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum.|

help_en

[+] Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor
    invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam
    et justo duo dolores et ea rebum.
[-] Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor
    invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam
    et justo duo dolores et ea rebum.

[jochenklar]

Looking at it again, can you set the padding of the areas with the text and the +/- to something like: 2px 8px 2px 8px and set the red green backround in the areas instead of outside?

[MyPyDavid]

I would also remove the red and green background (it conflicts with the border radius of all other panels buttons etc.). The plus and minus look like buttons, you could remove the border and add the light red and blue there.

We should also remove the "row/col indentation" from the displayed fields. Maybe we put the field name on top, and display either the diff or just the fields value, a bit like this:

help_de Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum.|

help_en

[+] Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor
    invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam
    et justo duo dolores et ea rebum.
[-] Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor
    invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam
    et justo duo dolores et ea rebum.

I think that showing only the diffs might be too confusing and maybe the user wants to sometimes copy/paste the values out of the page..

[jochenklar]

ok, makes sense. The regular output is also markdown rendered, right?

[MyPyDavid]

margin/paddings ok like this? PS ohw, now I see that the height of the [-] and text boxes are not aligned

WIP:

and

[MyPyDavid]

ok, makes sense. The regular output is also markdown rendered, right?

need to look into it, haven't changed anything to the regular output I think

[MyPyDavid]

• [x] think that the additional_input field on option still has problem, it should show [+] text here

[jochenklar]

I worked a bit on the css:

.field-diff {
    [class*="react-diff-"][class*="diff-container"] {
        margin: 0;
        border-collapse: separate;  // needed for border radius since this is a table

        tr {
            &:first-child {
                [class*="marker"] {
                    border-top-left-radius: 4px;
                }
                [class*="content"] {
                    border-top-right-radius: 4px;
                }
            }
            &:last-child {
                [class*="marker"] {
                    border-bottom-left-radius: 4px;
                }
                [class*="content"] {
                    border-bottom-right-radius: 4px;
                }
            }
        }

        // Target elements with classes containing "react-diff-", and "marker"
        // Target elements with classes containing "react-diff-", and "content"
        [class*="marker"],
        [class*="content"] {
            padding: 8px 12px;
            vertical-align: top;
            pre {
                padding: 5px 10px;
                border-radius: 4px;
                font-size: 13px;
                line-height: 18px;
                background-color: #fff;
            }
        }

        [class*="marker"] {
            padding-right: 0;
        }
    }
}

[jochenklar]

Can you change this

Select all
Select changed
Deselect all
Deselect changed

Same for show. (I think it is deselect not unselect.)

[jochenklar]

The margin-top from .import-buttons needs to go.

[jochenklar]

One more: I think the checkbox should be Show only created and changed and should also include the elements to be created. The .horizontal-container needs to have padding-bottom: 0;

[MyPyDavid]

Can you change this

Select all
Select changed
Deselect all
Deselect changed

Same for show. (I think it is deselect not unselect.)

I placed them like that because the "changed" is a subset of "all" but can also make it how you say

[jochenklar]

Yes, but the indent looks somehow "off". In the interview we have the same, but there is a moving arrow so there it makes sense.

[MyPyDavid]

I have one question about switching language/translations during import. It currently resets the page back to management (cancels the current import). How could I prevent this and just switch language during the import? Do I need to hook the imports.elements back into a reducer somewhere?

[jochenklar]

I think its fine, there would be no straight forward way to keep the state when the page is reloaded.

[MyPyDavid]

• [x] the URIs for Attributes are not properly built at import (as shown in the CodeLink )

• fixed

[MyPyDavid]

Front-end

• [x] update some things in the ImportSuccessElement
  • [x] pass importAction
  • [x] fix EditLink
• [x] fix the placement of Warnings and Errors inside the panels
• [x] handle created and add indicator
• [x] add AvailableLink toggle for Catalogs
• [x] show null, false, true or 0 (int) as strings in the import element panels
• [x] refactor the toggleImport checkbox into a component

Back-end

• [x] refactored string field names into enum constants
• [x] fixed ordering of elements
• [x] handle comparison for order field when '0' == 0
  • type cast int to str in this specific case

Other

• [ ] Attributes with key or path??
• [x] Import currently overwrites the comment field. How should this field be handled?
  • keep original comment when imported comment is empty
• the order of elements that are displayed is preserved from the xml file, when saving the elements are first ordered so that related elements without other relations are saved first.

[MyPyDavid]

I've implement the last todos (https://github.com/rdmorganiser/rdmo/pull/810#issuecomment-2155083683) and pasted in the current screenshot

Hopefully this is ready by now. Tests could still be extended for the m2m cases...

[MyPyDavid]

force pushed the interactive rebase on the current dev-2.2.0