liferay-portal icon indicating copy to clipboard operation
liferay-portal copied to clipboard

Published 20 hours ago • verified publisher icon brianchandotcom

LPD-22969 LAR import gets exponentially slow with many web contents interconnected through repeatable fields

Open liferay-continuous-integration opened this issue 10 months ago • 3 comments

Forwarded from: https://github.com/liferay-content-management/liferay-portal/pull/5363 (Took 1 ci:forward attempt in 1 hour 29 minutes) Console

@vendeltoreki @liferay-content-management

Original pull request comment: Forwarding from: https://github.com/liferay-echo/liferay-portal/pull/15548

See details: LPD-22969

Motivation

A fix in the past, LPSA-58080 (see commit) introduced a change that can cause performance issues when we are importing large amounts of web contents referencing each other.

Since that change, if not all of the web content field references could be resolved, the web content is removed from the "processed" list of the PortletDataContext. Thus, making the framework import it again, hoping that the references has been imported by then.

The problems this PR trying to solve:

  • Empty repeatable web content references were also considered missing.
  • Repeatable web content references are exported in a different way: they have classPK instead of articlePrimaryKey, but only articlePrimaryKey was considered.
  • If the web contents are too "interconnected", then the import will slow down exponentially, as all web contents are almost always removed from the processed list, and the dependencies are getting resolved very slowly, after thousands of iterations.

Proposed solution

  • Do a post-processing, after all web contents have been imported: https://github.com/liferay-content-management/liferay-portal/commit/259d94eba89e177d75de710e057246a217d4b61f
  • Try to resolve the references, even when they are coming from a repeatable web content field: https://github.com/liferay-content-management/liferay-portal/commit/07d7e9b62bd1d112728f5888839831eab766b89b

Please review my changes.

liferay-continuous-integration avatar Apr 23 '24 11:04 liferay-continuous-integration

:heavy_check_mark: ci:test:sf - 1 out of 1 jobs passed in 4 minutes

Click here for more details.

Base Branch:

Branch Name: master
Branch GIT ID: d56089f945aa90d7960108eba7f17186ac74963c

Sender Branch:

Branch Name: LPD-22969-3
Branch GIT ID: d63b1abf49dfacf416064d30c599ed87313c396f

1 out of 1jobs PASSED
1 Successful Jobs:
For more details click here.

liferay-continuous-integration avatar Apr 23 '24 11:04 liferay-continuous-integration

To conserve resources, the PR Tester does not automatically run for forwarded pull requests.

liferay-continuous-integration avatar Apr 23 '24 11:04 liferay-continuous-integration

:heavy_check_mark: ci:test:stable - 33 out of 33 jobs passed

:heavy_check_mark: ci:test:relevant - 47 out of 47 jobs passed in 1 hour 28 minutes

Click here for more details.

Base Branch:

Branch Name: master
Branch GIT ID: d56089f945aa90d7960108eba7f17186ac74963c

Upstream Comparison:

Branch GIT ID: d56089f945aa90d7960108eba7f17186ac74963c
Jenkins Build URL: EE Development Acceptance (master) - 501 - 2024-04-22[12:38:53]

ci:test:stable - 33 out of 33 jobs PASSED
33 Successful Jobs:
ci:test:relevant - 47 out of 47 jobs PASSED
47 Successful Jobs:
For more details click here.

liferay-continuous-integration avatar Apr 23 '24 11:04 liferay-continuous-integration

Merged. Thank you. View total diff: https://github.com/brianchandotcom/liferay-portal/compare/81773ff6e2...8a74634b63

brianchandotcom avatar Apr 27 '24 11:04 brianchandotcom

@vendeltoreki see 8a74634b63af6cda607ef81af8a7bdf478071ce1

brianchandotcom avatar Apr 27 '24 11:04 brianchandotcom