collective.exportimport icon indicating copy to clipboard operation
collective.exportimport copied to clipboard

Relations import, isReferencing is dropped in the target site after being filled by import_content?

Open fredvd opened this issue 3 years ago • 0 comments

Something funny is going on with the relations import.

  • I think I forgot to import relations in my target Plone 5.2 site. When I use @@inspect_relations from collective.relationhelpers I see around 1800 isReferencing relations in the target site.
  • So I exported relations again from the source Plone 4 site, import them in the target site.
  • Now my 1800 isReferencing relations are gone, but 800 relatesTo (related items) relations have been added.

In import_other isReferencing is added to the ignores list: https://github.com/collective/collective.exportimport/blob/5270f951355f6fbf119a7c6a7c2e8f249560001d/src/collective/exportimport/import_other.py#L271-L274

From the rest of the code in this method it seems all existing relations in the target site are removed and only the 'sanitised' relations are imported again. but this destroys all linkintegrity relations.

I assume the import_content imports are recreating the isReferencing relations while restoring the content items.

The most obvious fix would be to not drop the relations

in: https://github.com/collective/collective.exportimport/blob/5270f951355f6fbf119a7c6a7c2e8f249560001d/src/collective/exportimport/import_other.py#L287-L297

But this does not work, I get ObjectMissing Errors in z3c.relationfield.event.updateRelations where it tries to list existing relations:

[9] > /Users/fred/.buildout/eggs/cp38/z3c.relationfield-0.9.0-py3.8.egg/z3c/relationfield/event.py(81)updateRelations()
-> rels = list(catalog.findRelations({'from_id': obj_id}))
[10]   /Users/fred/.buildout/eggs/cp38/zc.relation-1.1.post2-py3.8.egg/zc/relation/catalog.py(734)<genexpr>()
-> return (self._relTools['load'](t, self, cache) for t in tokens)
[11]   /Users/fred/.buildout/eggs/cp38/z3c.relationfield-0.9.0-py3.8.egg/z3c/relationfield/index.py(49)load()
-> return intids.getObject(token)
[12]   /Users/fred/.buildout/eggs/cp38/zope.intid-4.3.0-py3.8.egg/zope/intid/__init__.py(89)getObject()
-> raise ObjectMissingError(id)

Stranger is that when I step back in the debugger to frame 9 and execute the same line again I do get the rels list:

(Pdb++) list(catalog.findRelations({'from_id': obj_id}))
[<z3c.relationfield.relation.RelationValue object at 0x11eefdac0 oid 0x200b0 in <Connection at 10d0bb760>>]

Or I could remove isReferencing from the ignores list

but then I'm importing the Plone 4 Archetypes linkintegrity lists on recreated Dexterity Content. It could work as no paths/ids etc have changed and for non existing content the relations will be dropped. But it doesn't feel very clean to do this.

[edit:] I tried this and the isReferencing is actually increasing when I include isReferencing relations on import. So importing content creates 1848 isReferencing relatins, then restoing relations ups isReferencing to 1941 relations. :-O

Export existing relations on the target site and merge them with the imported relations.

Meh. I mean doing this in the import_code: make an extra import on the fly, merge with the incoming relations.json and reapply, dropping isReferencing only from the relations.json .

@pbauer How did you deal with this so far while using collective.exportimport?

fredvd avatar May 20 '21 13:05 fredvd