[16.0][ADD] dms_import: Migration data from documents EE to dms CE

[xaviedoanhduy]

Purpose

This migration script allows moving data from Enterprise documents* (EE) to the OCA dms* (CE) modules.

The goal is to preserve:

• Folder hierarchy
• Access rights (read/write groups)
• Tags and tag categories
• Files and their attachments

---

Approach

The migration is implemented as a post-init hook and works directly at the SQL level to avoid dependencies on EE models. This ensures:

• Compatibility with CE environments (no EE models required).
• Preservation of hierarchy, security, and data.

The migration flow has three steps:

1. Tags migration

• documents.facet → dms.category
• documents.tag → dms.tag
• Prevents duplicates by checking existing categories and tags.

2. Folders migration

• documents.folder → dms.directory
• Maintains parent/child relationships.
• Root folders receive a default dms.storage.
• Access rights migrated via:
  • documents_folder_res_groups_rel → dms.access.group (Write groups)
  • documents_folder_read_groups_rel → dms.access.group (Read groups)
• Folder-level tags are also migrated via facet → tag mapping.

3. Files migration

• documents.document (binary only) → dms.file
• Preserves folder assignment and file metadata.
• Keeps linked attachments by updating ir.attachment.res_model/res_id to the new dms.file.
• Migrates file tags using the tag mapping.
• Uses batch processing (1000 docs per batch) for scalability.

---

Data Mapping Details

EE Model / Table	DMS Model / Table	Notes
documents.facet	dms.category	Folder tag categories → Categories
documents.tag	dms.tag	Tag names + facet → Tags, with deduplication
documents.folder	dms.directory	Folder hierarchy → Directory hierarchy, sequence → color (fallback)
documents_folder_res_groups_rel	dms.access.group	Write group permissions (create/write/unlink)
documents_folder_read_groups_rel	dms.access.group	Read-only group permissions
documents.folder.facet_ids.tag_ids	dms.directory.tag_ids	Folder-level tags migrated via mapping
documents.document (binary)	dms.file	Binary documents only, skipped inactive or non-binary docs
documents_document_tag_rel	dms_file_tag_rel	Many2many doc ↔ tags mapping
ir.attachment (linked to document)	ir.attachment (relinked)	Updated to point to new dms.file

---

Extra Notes

• Colors default to randint(1, 11) when no valid sequence color is available.
• A default database storage (dms.storage) is created if none exists.
• Logging is verbose:
  • Created directories
  • Assigned groups
  • Migrated tags and files
• Errors are logged per record without blocking the migration.

---

Example Workflow

1. Run module installation with the migration hook.
2. Check that:
   • Categories and tags exist in dms.category and dms.tag.
   • Directories are created with correct hierarchy.
   • Access groups are properly assigned.
   • Files are migrated and linked to attachments.
   • Tags are assigned to both directories and files.

---

Validation

• Before migration: Ensure EE tables (documents_*) exist and contain data.
• After migration: Validate:
  • Number of categories, tags, and directories.
  • Randomly sample files to confirm attachments and tags.
  • Check group permissions are correctly migrated.

---

Migration should complete successfully and log a summary:

Successfully migrated X tags.
Successfully migrated Y folders (Z new root directories).
Successfully migrated N files.
Migration completed successfully.

[kobros-tech]

@xaviedoanhduy

could you write desctiption on how to use it or configure if necessary, so that we could review easily?

[nilshamerlinck]

hi @kobros-tech you will receive an invitation to join the slack workspace where you'll find more information, see you there ;-)

[kobros-tech]

@nilshamerlinck thanks, I have now a stack account!

[kobros-tech]

@xaviedoanhduy

what can happen if there is a file that should be for a specific partner, shall we get the same file accessible by the same partner Only after migration?

[kobros-tech]

all right I will ping other mind to tell about real life scenario and then we can apply

@wlin-kencove

[xaviedoanhduy]

@kobros-tech,

As far as I recall, dms.file/dms.directory are shared at the user - res.users level, while documents.document/documents.folder are assigned to contacts - res.partner.

I’d appreciate your thoughts if you have any ideas on how to map these fields between the two models.

Sorry for my omission, I'd like to clarify some points:

• For your question, it seems that the answer is no. because the data from EE documents is not shared with any user (it can only be owned by 1 partner) but only through links or permission groups - I will give an example that the Internal folder has Read Groups (Write Groups) containing the Documents/Adminstrator permission group and the user named Demo is not in this group, he will not be able to see the above folder in the backend view - but he can have full read and write permissions if he owns the link generated from Share links.
• EE documents, the documents will mostly not be accessible flexibly to portal users (even if that user is the owner of that document - it will only be seen through the backend in the Contact app). And for portal users (or even non-logged in users), they only need to know the 1 link created from Share links (documents.share model) - and that is the only way they can access these documents.
• For CE dms, these sensitive records are controlled from another group model (dms.access.group) and allow all public users to see these documents if their group (res.groups - portal group) or maybe on users (res.users).
• For the context of the current module, I am using the mechanism of creating dms.access.groups based on the name of the document and Read Groups (Write Groups) -> resulting in only users in the above permission groups can access it - and one thing is for sure, those permission groups do not contain portal users.

[kobros-tech]

yes, it is much better.

once you are done you can ask me to review, good luck!

[andreampiovesana]

nice to have