magento
[2.4.5] setup:static-content:deploy hanging after upgrading
Preconditions and environment
- Magento 2.4.5
- PHP 8.1
Steps to reproduce
- Run
bin/magento setup:static-content:deploy -f -j 4 en_US en_GB nl_NL nl_BE sv_SE
Expected result
Execution time is the same as 2.4.4:
❯ time php -d memory_limit=-1 bin/magento s:s:d -vvvvvv -f -j 4 en_GB nl_BE nl_NL da_DK sv_SE
[...]Progress bars[...]
Execution time: 45.576536178589
php -d memory_limit=-1 bin/magento s:s:d -vvvvvv -f -j 4 en_GB nl_BE nl_NL 99.31s user 26.01s system 266% cpu 46.942 total
Actual result
Process appears to hang and is using 100% of the CPU core it's running on, any child processes that were created will be marked as Zombie since the parent is stuck and cannot reap them.
In some cases, the execution time of s:s:d is increased from 45 seconds to over 45 minutes.
As seen in the screenshot, the process is already running for over 10 minutes while normally this would take 45 seconds. The 2 sub-processes are marked with Z (zombie) as the main thread is stuck in a loop in \Magento\Deploy\Package\Processor\PreProcessor\Less::inParentFiles.
Additional information
Reverting the change made in https://github.com/magento/magento2/commit/4514e651f5e793cbee6123487b72aafa119bd1fe will restore the static content deploy quick strategy performance and reduce the execution time back to normal levels.
It seems that every iteration through \Magento\Deploy\Package\Processor\PreProcessor\Less::inParentFiles will increase the deployment time until it reaches a point where it gets completely stuck in the loop.
As it stands, this is blocking our server deployments and a revert-patch is needed to make them succeed.
diff --git a/htdocs/vendor/magento/module-deploy/Package/Processor/PreProcessor/Less.php b/htdocs/vendor/magento/module-deploy/Package/Processor/PreProcessor/Less.php
--- a/htdocs/vendor/magento/module-deploy/Package/Processor/PreProcessor/Less.php
+++ b/htdocs/vendor/magento/module-deploy/Package/Processor/PreProcessor/Less.php
@@ -154,9 +154,7 @@
return true;
} else {
foreach ($map[$parentFile] as $pFile) {
- if ($this->inParentFiles($fileName, $pFile, $map)) {
- return true;
- }
+ return $this->inParentFiles($fileName, $pFile, $map);
}
}
}
Release note
No response
Triage and priority
- [ ] Severity: S0 - Affects critical data or functionality and leaves users without workaround.
- [X] Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
- [ ] Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
- [ ] Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
- [ ] Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.
Hi @DuckThom. Thank you for your report. To speed up processing of this issue, make sure that you provided the following information:
- Summary of the issue
- Information on your environment
- Steps to reproduce
- Expected and actual results
Make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, Add a comment to the issue:
@magento give me 2.4-develop instance - upcoming 2.4.x release
For more details, review the Magento Contributor Assistant documentation.
Add a comment to assign the issue: @magento I am working on this
To learn more about issue processing workflow, refer to the Code Contributions.
- Join Magento Community Engineering Slack and ask your questions in #github channel.
:warning: According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.
:clock10: You can find the schedule on the Magento Community Calendar page.
:telephone_receiver: The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.
:pencil2: Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel
![m2-assistant[bot] avatar](https://avatars.githubusercontent.com/in/27048?v=4 "Published by a pub.dev verified publisher"){kind=small}
+1 I can confirm severe performance degradation in static content deploy on three independent projects, all using a quick deploy strategy using multiple jobs in parallel. All these projects are being upgraded to 2.4.5 and this issue started occurring when deploying our upgrade branch to test and acceptance environments. Production is on 2.4.3-p2 for all 3 projects and is unaffected by performance degradation.
Edit: below are some output logs to show the severity of the degradation between 2.4.3-p2 and 2.4.5.
Production deployment (Magento 2.4.3-p2 + PHP7.4 - no performance degradation)
frontend/Magento/blank/en_GB 3425/3425 ============================ 100% 16 secs
frontend/Magento/blank/da_DK 3425/3425 ============================ 100% 13 secs
frontend/Magento/blank/nl_BE 3425/3425 ============================ 100% 20 secs
frontend/Magento/blank/fr_BE 3425/3425 ============================ 100% 18 secs
frontend/Magento/blank/de_DE 3425/3425 ============================ 100% 14 secs
frontend/Magento/blank/de_AT 3425/3425 ============================ 100% 13 secs
frontend/Magento/blank/nb_NO 3425/3425 ============================ 100% 14 secs
frontend/Magento/blank/de_CH 3425/3425 ============================ 100% 13 secs
frontend/Magento/blank/fr_CH 3425/3425 ============================ 100% 19 secs
frontend/Magento/blank/it_CH 3425/3425 ============================ 100% 19 secs
frontend/Magento/blank/en_IE 3425/3425 ============================ 100% 19 secs
frontend/Magento/blank/ar_AE 3425/3425 ============================ 100% 13 secs
frontend/Magento/blank/zh_Hant_HK 3425/3425 ============================ 100% 16 secs
frontend/Magento/blank/en_HK 3425/3425 ============================ 100% 13 secs
frontend/Magento/blank/it_IT 3425/3425 ============================ 100% 15 secs
frontend/Magento/blank/es_ES 3425/3425 ============================ 100% 13 secs
frontend/ClientTheme/base/da_DK 3654/3654 ============================ 100% 25 secs
frontend/ClientTheme/base/nl_NL 3654/3654 ============================ 100% 24 secs
frontend/ClientTheme/base/en_GB 3654/3654 ============================ 100% 22 secs
frontend/ClientTheme/base/nl_BE 3654/3654 ============================ 100% 22 secs
frontend/ClientTheme/base/fr_BE 3654/3654 ============================ 100% 29 secs
frontend/ClientTheme/base/de_DE 3654/3654 ============================ 100% 29 secs
frontend/ClientTheme/base/de_AT 3654/3654 ============================ 100% 22 secs
frontend/ClientTheme/base/de_CH 3654/3654 ============================ 100% 22 secs
frontend/ClientTheme/base/fr_CH 3654/3654 ============================ 100% 25 secs
frontend/ClientTheme/base/it_CH 3654/3654 ============================ 100% 25 secs
frontend/ClientTheme/base/en_IE 3654/3654 ============================ 100% 23 secs
frontend/ClientTheme/base/ar_AE 3654/3654 ============================ 100% 23 secs
frontend/ClientTheme/base/zh_Hant_HK 3654/3654 ============================ 100% 28 secs
frontend/ClientTheme/base/en_HK 3654/3654 ============================ 100% 26 secs
frontend/ClientTheme/base/it_IT 3654/3654 ============================ 100% 27 secs
frontend/ClientTheme/base/es_ES 3654/3654 ============================ 100% 25 secs
Acceptance deployment (Magento 2.4.5 + PHP8.1 - performance degradation)
frontend/Magento/blank/en_GB 3424/3424 ============================ 100% 15 secs
frontend/Magento/blank/da_DK 3424/3424 ============================ 100% 19 secs
frontend/Magento/blank/nl_BE 3424/3424 ============================ 100% 16 secs
frontend/Magento/blank/fr_BE 3424/3424 ============================ 100% 15 secs
frontend/Magento/blank/de_DE 3424/3424 ============================ 100% 16 secs
frontend/Magento/blank/de_AT 3424/3424 ============================ 100% 18 secs
frontend/Magento/blank/nb_NO 3424/3424 ============================ 100% 15 secs
frontend/Magento/blank/de_CH 3424/3424 ============================ 100% 17 secs
frontend/Magento/blank/fr_CH 3424/3424 ============================ 100% 18 secs
frontend/Magento/blank/it_CH 3424/3424 ============================ 100% 15 secs
frontend/Magento/blank/en_IE 3424/3424 ============================ 100% 17 secs
frontend/Magento/blank/ar_AE 3424/3424 ============================ 100% 15 secs
frontend/Magento/blank/zh_Hant_HK 3424/3424 ============================ 100% 21 secs
frontend/Magento/blank/en_HK 3424/3424 ============================ 100% 15 secs
frontend/Magento/blank/it_IT 3424/3424 ============================ 100% 16 secs
frontend/Magento/blank/es_ES 3424/3424 ============================ 100% 19 secs
frontend/ClientTheme/base/da_DK 3653/3653 ============================ 100% 25 secs
frontend/ClientTheme/base/nl_NL 3653/3653 ============================ 100% 23 secs
frontend/ClientTheme/base/en_GB 3653/3653 ============================ 100% 28 secs
frontend/ClientTheme/base/nl_BE 3653/3653 ============================ 100% 27 secs
frontend/ClientTheme/base/fr_BE 3653/3653 ============================ 100% 31 secs
frontend/ClientTheme/base/de_DE 3653/3653 ============================ 100% 33 secs
frontend/ClientTheme/base/de_AT 3653/3653 ============================ 100% 44 secs
frontend/ClientTheme/base/de_CH 3653/3653 ============================ 100% 51 secs
frontend/ClientTheme/base/fr_CH 3653/3653 ============================ 100% 1 min
frontend/ClientTheme/base/it_CH 3653/3653 ============================ 100% 1 min
frontend/ClientTheme/base/en_IE 3653/3653 ============================ 100% 2 mins
frontend/ClientTheme/base/ar_AE 3653/3653 ============================ 100% 2 mins
frontend/ClientTheme/base/zh_Hant_HK 3653/3653 ============================ 100% 3 mins
frontend/ClientTheme/base/en_HK 3653/3653 ============================ 100% 4 mins
frontend/ClientTheme/base/it_IT 3653/3653 ============================ 100% 5 mins
frontend/ClientTheme/base/es_ES 3653/3653 ============================ 100% 7 mins
Hi @engcom-Delta. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:
-
[ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
Details
If the issue has a valid description, the labelIssue: Format is validwill be added to the issue automatically. Please, edit issue description if needed, until labelIssue: Format is validappears. -
[ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add
Issue: Clear Descriptionlabel to the issue by yourself. -
[ ] 3. Add
Component: XXXXXlabel(s) to the ticket, indicating the components it may be related to. -
[ ] 4. Verify that the issue is reproducible on
2.4-developbranchDetails
- Add the comment@magento give me 2.4-develop instanceto deploy test instance on Magento infrastructure.
- If the issue is reproducible on2.4-developbranch, please, add the labelReproduced on 2.4.x.
- If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here! -
[ ] 5. Add label
Issue: Confirmedonce verification is complete. -
[ ] 6. Make sure that automatic system confirms that report has been added to the backlog.
Hi @DuckThom,
Thank you for reporting and collaboration.
We have tried to reproduce this issue but issue is not reproducible .
Precondition: • Magento version : 2.4.5-develop • PHP 8.1
We have followed below steps 1.Ran bin/magento setup:static-content:deploy -f -j 4 en_US en_GB nl_NL nl_BE sv_SE in magento 2.4.4
2.After upgrading Magento2.4.4 to Magento2.4.5 When we ran the bin/magento setup:static-content:deploy -f -j 4 en_US en_GB nl_NL nl_BE sv_SE we are getting as below ,please have a look .
- we are not able to see process appears to hang
Please let us know if we are missing any steps here.
Thanks
Hi @engcom-Delta
It’s possible that on a default installation, there's not enough files yet for it to become noticably slower. I do see in your screenshot that the 2.4.5 build took 55s where the 2.4.4 build took 40s.
On our projects we have larger themes installed with some third party dependencies so the internal map it’s building in the Less preprocessor will be much larger which will make the slowdown much more noticeable.
I’ll see if I can find a way to more accurately reproduce this on a clean installation.
I've just tested it out over here on a project where we are upgrading from Magento 2.3.7-p4 to Magento 2.4.5 and can confirm that the same problem is happening. When I switch from 4 jobs to 1 job, it's no longer hanging, but it takes a lot longer to execute then normal.
With 4 jobs, it just seems stuck on the line for frontend/Magento/luma/sv_SE:
We have 2 custom themes where one has parent: Magento/blank and the other has the other custom theme as parent.
@engcom-Delta: do you have the php extension pcntl installed and active (you can check with php -m | grep pcntl)? Because you'll need it before the parallel jobs will start working. Also make sure to always execute the following between test runs: rm -R var/view_preprocessed/* pub/static/*.
@engcom-Delta I've done some testing on clean installations and the slowdown effect is not very noticeable, however it is still there. Here are some timings taken from my testing:
Magento Open Source
2.4.4
❯ rm -rf var/view_preprocessed pub/static/* && php -d memory_limit=-1 bin/magento s:s:d -f -j 4 en_GB nl_BE nl_NL da_DK sv_SE
Deploy using quick strategy
frontend/Magento/blank/en_GB 2174/2174 ============================ 100% < 1 sec
adminhtml/Magento/backend/en_GB 2911/2911 ============================ 100% < 1 sec
frontend/Magento/blank/nl_BE 2174/2174 ============================ 100% < 1 sec
frontend/Magento/blank/nl_NL 2174/2174 ============================ 100% < 1 sec
frontend/Magento/blank/da_DK 2174/2174 ============================ 100% 2 secs
frontend/Magento/blank/sv_SE 2174/2174 ============================ 100% 2 secs
adminhtml/Magento/backend/nl_BE 2911/2911 ============================ 100% 2 secs
adminhtml/Magento/backend/nl_NL 2911/2911 ============================ 100% 1 sec
adminhtml/Magento/backend/da_DK 2911/2911 ============================ 100% 0
adminhtml/Magento/backend/sv_SE 2911/2911 ============================ 100% 0
frontend/Magento/luma/en_GB 2190/2190 ============================ 100% 0
frontend/Magento/luma/nl_BE 2190/2190 ============================ 100% 3 secs
frontend/Magento/luma/nl_NL 2190/2190 ============================ 100% 1 sec
frontend/Magento/luma/da_DK 2190/2190 ============================ 100% 1 sec
frontend/Magento/luma/sv_SE 2190/2190 ============================ 100% < 1 sec
Execution time: 19.21523809433
2.4.5
❯ rm -rf var/view_preprocessed pub/static/* && php -d memory_limit=-1 bin/magento s:s:d -f -j 4 en_GB nl_BE nl_NL da_DK sv_SE
Deploy using quick strategy
frontend/Magento/blank/en_GB 2181/2181 ============================ 100% < 1 sec
adminhtml/Magento/backend/en_GB 2921/2921 ============================ 100% < 1 sec
frontend/Magento/blank/nl_BE 2181/2181 ============================ 100% < 1 sec
frontend/Magento/blank/nl_NL 2181/2181 ============================ 100% < 1 sec
frontend/Magento/blank/da_DK 2181/2181 ============================ 100% 2 secs
frontend/Magento/blank/sv_SE 2181/2181 ============================ 100% 2 secs
adminhtml/Magento/backend/nl_BE 2921/2921 ============================ 100% 2 secs
adminhtml/Magento/backend/nl_NL 2921/2921 ============================ 100% 1 sec
adminhtml/Magento/backend/da_DK 2921/2921 ============================ 100% 0
adminhtml/Magento/backend/sv_SE 2921/2921 ============================ 100% 0
frontend/Magento/luma/en_GB 2197/2197 ============================ 100% 0
frontend/Magento/luma/nl_BE 2197/2197 ============================ 100% 0
frontend/Magento/luma/nl_NL 2197/2197 ============================ 100% 3 secs
frontend/Magento/luma/da_DK 2197/2197 ============================ 100% 3 secs
frontend/Magento/luma/sv_SE 2197/2197 ============================ 100% 2 secs
Execution time: 21.784950017929
As you can see, not a large difference. Things get more interesting though when we switch to a clean enterprise installation (since it has an extra theme in 2.4.5, Magento/spectrum)
Magento Commerce
2.4.4
❯ rm -rf var/view_preprocessed pub/static/* && php -d memory_limit=-1 bin/magento s:s:d -f -j 4 en_GB nl_BE nl_NL da_DK sv_SE
Deploy using quick strategy
frontend/Magento/blank/en_GB 2303/2303 ============================ 100% < 1 sec
adminhtml/Magento/backend/en_GB 3000/3000 ============================ 100% < 1 sec
frontend/Magento/blank/nl_BE 2303/2303 ============================ 100% < 1 sec
frontend/Magento/blank/nl_NL 2303/2303 ============================ 100% < 1 sec
frontend/Magento/blank/da_DK 2303/2303 ============================ 100% 2 secs
frontend/Magento/blank/sv_SE 2303/2303 ============================ 100% 2 secs
adminhtml/Magento/backend/nl_BE 3000/3000 ============================ 100% 2 secs
adminhtml/Magento/backend/nl_NL 3000/3000 ============================ 100% 1 sec
adminhtml/Magento/backend/da_DK 3000/3000 ============================ 100% 0
adminhtml/Magento/backend/sv_SE 3000/3000 ============================ 100% 0
frontend/Magento/luma/en_GB 2320/2320 ============================ 100% 3 secs
frontend/Magento/luma/nl_BE 2320/2320 ============================ 100% 3 secs
frontend/Magento/luma/nl_NL 2320/2320 ============================ 100% < 1 sec
frontend/Magento/luma/da_DK 2320/2320 ============================ 100% < 1 sec
frontend/Magento/luma/sv_SE 2320/2320 ============================ 100% 0
Execution time: 20.095071077347
2.4.5
❯ rm -rf var/view_preprocessed pub/static/* && php -d memory_limit=-1 bin/magento s:s:d -f -j 4 en_GB nl_BE nl_NL da_DK sv_SE
Deploy using quick strategy
frontend/Magento/blank/en_GB 2318/2318 ============================ 100% < 1 sec
adminhtml/Magento/backend/en_GB 3012/3012 ============================ 100% < 1 sec
frontend/Magento/blank/nl_BE 2318/2318 ============================ 100% < 1 sec
frontend/Magento/blank/nl_NL 2318/2318 ============================ 100% < 1 sec
frontend/Magento/blank/da_DK 2318/2318 ============================ 100% 2 secs
frontend/Magento/blank/sv_SE 2318/2318 ============================ 100% 2 secs
adminhtml/Magento/backend/nl_BE 3012/3012 ============================ 100% 2 secs
adminhtml/Magento/backend/nl_NL 3012/3012 ============================ 100% 1 sec
adminhtml/Magento/backend/da_DK 3012/3012 ============================ 100% 0
adminhtml/Magento/backend/sv_SE 3012/3012 ============================ 100% 0
frontend/Magento/luma/en_GB 2335/2335 ============================ 100% 0
adminhtml/Magento/spectrum/en_GB 3012/3012 ============================ 100% 0
frontend/Magento/luma/nl_BE 2335/2335 ============================ 100% 0
frontend/Magento/luma/nl_NL 2335/2335 ============================ 100% 3 secs
frontend/Magento/luma/da_DK 2335/2335 ============================ 100% 2 secs
frontend/Magento/luma/sv_SE 2335/2335 ============================ 100% 0
adminhtml/Magento/spectrum/nl_BE 3012/3012 ============================ 100% 0
adminhtml/Magento/spectrum/nl_NL 3012/3012 ============================ 100% 0
adminhtml/Magento/spectrum/da_DK 3012/3012 ============================ 100% 0
adminhtml/Magento/spectrum/sv_SE 3012/3012 ============================ 100% 0
Execution time: 80.963579893112
As you can see, the execution time increased from 20s to 80s, a 300% increase in compilation time. Now, an increase is obviously expected when a new theme is added, however, a 3x increase seems excessive.
Using this information, I decided to grab a random theme from the marketplace to see how the times would change in Magento Open Source since virtually all Magento sites are using one or more custom themes.
Magento Open Source
2.4.4
❯ rm -rf var/view_preprocessed pub/static/\* && php -d memory_limit=-1 bin/magento s:s:d -f -j 4 en_GB nl_BE nl_NL da_DK sv_SE
Deploy using quick strategy
frontend/Magento/blank/en_GB 2210/2210 ============================ 100% < 1 sec
adminhtml/Magento/backend/en_GB 2971/2971 ============================ 100% < 1 sec
frontend/Magento/blank/nl_BE 2210/2210 ============================ 100% < 1 sec
frontend/Magento/blank/nl_NL 2210/2210 ============================ 100% < 1 sec
frontend/Magento/blank/da_DK 2210/2210 ============================ 100% 0
frontend/Magento/blank/sv_SE 2210/2210 ============================ 100% 0
adminhtml/Magento/backend/nl_BE 2971/2971 ============================ 100% 0
adminhtml/Magento/backend/nl_NL 2971/2971 ============================ 100% 0
adminhtml/Magento/backend/da_DK 2971/2971 ============================ 100% 0
adminhtml/Magento/backend/sv_SE 2971/2971 ============================ 100% 0
frontend/Magento/luma/en_GB 2226/2226 ============================ 100% 0
frontend/Magento/luma/nl_BE 2226/2226 ============================ 100% 0
frontend/Magento/luma/nl_NL 2226/2226 ============================ 100% 0
frontend/Magento/luma/da_DK 2226/2226 ============================ 100% 0
frontend/Magento/luma/sv_SE 2226/2226 ============================ 100% 0
frontend/Swissup/absolute/en_GB 2261/2261 ============================ 100% 0
frontend/Swissup/absolute/nl_BE 2261/2261 ============================ 100% 0
frontend/Swissup/absolute/nl_NL 2261/2261 ============================ 100% 0
frontend/Swissup/absolute/da_DK 2261/2261 ============================ 100% 0
frontend/Swissup/absolute/sv_SE 2261/2261 ============================ 100% 0
Execution time: 7.2641670703888
2.4.5
❯ rm -rf var/view_preprocessed pub/static/* && php -d memory_limit=-1 bin/magento s:s:d -f -j 4 en_GB nl_BE nl_NL da_DK sv_SE
Deploy using quick strategy
frontend/Magento/blank/en_GB 2217/2217 ============================ 100% < 1 sec
adminhtml/Magento/backend/en_GB 2981/2981 ============================ 100% < 1 sec
frontend/Magento/blank/nl_BE 2217/2217 ============================ 100% < 1 sec
frontend/Magento/blank/nl_NL 2217/2217 ============================ 100% < 1 sec
frontend/Magento/blank/da_DK 2217/2217 ============================ 100% 2 secs
frontend/Magento/blank/sv_SE 2217/2217 ============================ 100% 2 secs
adminhtml/Magento/backend/nl_BE 2981/2981 ============================ 100% 2 secs
adminhtml/Magento/backend/nl_NL 2981/2981 ============================ 100% 1 sec
adminhtml/Magento/backend/da_DK 2981/2981 ============================ 100% 0
adminhtml/Magento/backend/sv_SE 2981/2981 ============================ 100% 0
frontend/Magento/luma/en_GB 2233/2233 ============================ 100% 0
frontend/Magento/luma/nl_BE 2233/2233 ============================ 100% 0
frontend/Magento/luma/nl_NL 2233/2233 ============================ 100% 0
frontend/Magento/luma/da_DK 2233/2233 ============================ 100% 0
frontend/Magento/luma/sv_SE 2233/2233 ============================ 100% 0
frontend/Swissup/absolute/en_GB 2268/2268 ============================ 100% 4 secs
frontend/Swissup/absolute/nl_BE 2268/2268 ============================ 100% 0
frontend/Swissup/absolute/nl_NL 2268/2268 ============================ 100% 0
frontend/Swissup/absolute/da_DK 2268/2268 ============================ 100% 0
frontend/Swissup/absolute/sv_SE 2268/2268 ============================ 100% 0
Execution time: 93.378259181976
And this time, the execution time also increased dramatically on the Open Source installation. Where it was 7s in 2.4.4, it increased massively to 93s in 2.4.5 using only a base installation with 1 custom theme and no additional code or changes. Combine this with the added admin theme in Commerce and the process execution time will increase exponentially.
I hope this will be of use to you.
It looks like this might be fixed by https://github.com/magento/magento2/commit/12a9eacaf57baf1dbd71b22cbd9ada32f3ffa479
I just applied these changes to my 2.4.5 Open Source testing installation I used above (where the compilation time was 93s) and it's now back to a more normal 26s
Unfortunately, it doesn't look like this will be released with 2.4.5-p1 as I just installed the pre-release in my Commerce testing installation and this commit is not in there.
It can, however, be installed as a patch by applying the changes from these commits (in order):
- https://github.com/magento/magento2/commit/8635f6b3ab2697a4dc1904dc16adc9c37518dcdf
- https://github.com/magento/magento2/commit/571ad3019cac39aa2822e20108516da5d20b84f5
- https://github.com/magento/magento2/commit/79ef3d9c087d61ad31124abdbee5326a42299681
- https://github.com/magento/magento2/commit/12a9eacaf57baf1dbd71b22cbd9ada32f3ffa479
Patch
View
From 8635f6b3ab2697a4dc1904dc16adc9c37518dcdf Mon Sep 17 00:00:00 2001
From: Roman Flowers <[email protected]>
Date: Tue, 19 Jul 2022 13:25:36 -0500
Subject: [PATCH] ACP2E-923: Compact strategy causes an invalid reference to an
external domain when running SCD
---
.../Package/Processor/PreProcessor/Css.php | 23 +++++++++++++++----
1 file changed, 19 insertions(+), 4 deletions(-)
diff --git a/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php b/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php
index 152c95f86552c..c65e6a4e9f78d 100644
--- a/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php
+++ b/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php
@@ -120,6 +120,19 @@ private function hasOverrides(PackageFile $parentFile, Package $package)
return false;
}
+ /**
+ * See if given path is local or remote URL
+ *
+ * @param string $path
+ * @return bool
+ */
+ private function isLocal(string $path): bool {
+ $pattern = '{^(file://(?!//)|/(?!/)|/?[a-z]:[\\\\/]|\.\.[\\\\/]|[a-z0-9_.-]+[\\\\/])}i';
+ $result = preg_match($pattern, $path);
+
+ return (bool) $result;
+ }
+
/**
* Build map file
*
@@ -138,10 +151,12 @@ private function buildMap($packagePath, $filePath, $fullPath)
$content = $this->staticDir->readFile($this->minification->addMinifiedSign($fullPath));
$callback = function ($matchContent) use ($packagePath, $filePath, &$imports) {
- $importRelPath = $this->normalize(pathinfo($filePath, PATHINFO_DIRNAME) . '/' . $matchContent['path']);
- $imports[$importRelPath] = $this->normalize(
- $packagePath . '/' . pathinfo($filePath, PATHINFO_DIRNAME) . '/' . $matchContent['path']
- );
+ if ($this->isLocal($matchContent['path'])) {
+ $importRelPath = $this->normalize(pathinfo($filePath, PATHINFO_DIRNAME) . '/' . $matchContent['path']);
+ $imports[$importRelPath] = $this->normalize(
+ $packagePath . '/' . pathinfo($filePath, PATHINFO_DIRNAME) . '/' . $matchContent['path']
+ );
+ }
};
preg_replace_callback(Import::REPLACE_PATTERN, $callback, $content);
From 571ad3019cac39aa2822e20108516da5d20b84f5 Mon Sep 17 00:00:00 2001
From: Roman Flowers <[email protected]>
Date: Wed, 20 Jul 2022 07:34:52 -0500
Subject: [PATCH] ACP2E-923: Compact strategy causes an invalid reference to an
external domain when running SCD
---
.../Magento/Deploy/Package/Processor/PreProcessor/Css.php | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php b/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php
index c65e6a4e9f78d..cd05109ce6300 100644
--- a/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php
+++ b/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php
@@ -126,7 +126,8 @@ private function hasOverrides(PackageFile $parentFile, Package $package)
* @param string $path
* @return bool
*/
- private function isLocal(string $path): bool {
+ private function isLocal(string $path): bool
+ {
$pattern = '{^(file://(?!//)|/(?!/)|/?[a-z]:[\\\\/]|\.\.[\\\\/]|[a-z0-9_.-]+[\\\\/])}i';
$result = preg_match($pattern, $path);
@@ -152,7 +153,9 @@ private function buildMap($packagePath, $filePath, $fullPath)
$callback = function ($matchContent) use ($packagePath, $filePath, &$imports) {
if ($this->isLocal($matchContent['path'])) {
- $importRelPath = $this->normalize(pathinfo($filePath, PATHINFO_DIRNAME) . '/' . $matchContent['path']);
+ $importRelPath = $this->normalize(
+ pathinfo($filePath, PATHINFO_DIRNAME) . '/' . $matchContent['path']
+ );
$imports[$importRelPath] = $this->normalize(
$packagePath . '/' . pathinfo($filePath, PATHINFO_DIRNAME) . '/' . $matchContent['path']
);
From 79ef3d9c087d61ad31124abdbee5326a42299681 Mon Sep 17 00:00:00 2001
From: Roman Flowers <[email protected]>
Date: Wed, 20 Jul 2022 12:44:23 -0500
Subject: [PATCH] ACP2E-923: Compact strategy causes an invalid reference to an
external domain when running SCD
---
vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php b/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php
index cd05109ce6300..8799cb62272c3 100644
--- a/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php
+++ b/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php
@@ -131,7 +131,7 @@ private function isLocal(string $path): bool
$pattern = '{^(file://(?!//)|/(?!/)|/?[a-z]:[\\\\/]|\.\.[\\\\/]|[a-z0-9_.-]+[\\\\/])}i';
$result = preg_match($pattern, $path);
- return (bool) $result;
+ return is_int($result) ? (bool) $result : true;
}
/**
From 12a9eacaf57baf1dbd71b22cbd9ada32f3ffa479 Mon Sep 17 00:00:00 2001
From: Oleksandr Iegorov <[email protected]>
Date: Fri, 9 Sep 2022 09:31:20 -0500
Subject: [PATCH] ACP2E-1168: Static content deployment timed out or failed
---
vendor/magento/module-deploy/Collector/Collector.php | 3 +++
.../Deploy/Package/Processor/PreProcessor/Css.php | 7 ++++++-
.../Deploy/Package/Processor/PreProcessor/Less.php | 10 ++++++++++
.../testsuite/Magento/Deploy/DeployTest.php | 2 +-
4 files changed, 20 insertions(+), 2 deletions(-)
diff --git a/vendor/magento/module-deploy/Collector/Collector.php b/vendor/magento/module-deploy/Collector/Collector.php
index b09001a7ac04c..441d165f6792a 100644
--- a/vendor/magento/module-deploy/Collector/Collector.php
+++ b/vendor/magento/module-deploy/Collector/Collector.php
@@ -93,6 +93,9 @@ public function collect()
if ($file->getModule() && !$this->moduleManager->isEnabled($file->getModule())) {
continue;
}
+ if (!$file->getFileName()) {
+ continue;
+ }
$file->setDeployedFileName($this->fileNameResolver->resolve($file->getFileName()));
$params = $this->getParams($file);
$packagePath = "{$params['area']}/{$params['theme']}/{$params['locale']}";
diff --git a/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php b/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php
index 8799cb62272c3..6fc9c05eb3527 100644
--- a/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php
+++ b/vendor/magento/module-deploy/Package/Processor/PreProcessor/Css.php
@@ -149,7 +149,12 @@ private function buildMap($packagePath, $filePath, $fullPath)
$imports = [];
$this->map[$fullPath] = [];
- $content = $this->staticDir->readFile($this->minification->addMinifiedSign($fullPath));
+ $tmpFilename = $this->minification->addMinifiedSign($fullPath);
+ if ($this->staticDir->isReadable($tmpFilename)) {
+ $content = $this->staticDir->readFile($tmpFilename);
+ } else {
+ $content = '';
+ }
$callback = function ($matchContent) use ($packagePath, $filePath, &$imports) {
if ($this->isLocal($matchContent['path'])) {
diff --git a/vendor/magento/module-deploy/Package/Processor/PreProcessor/Less.php b/vendor/magento/module-deploy/Package/Processor/PreProcessor/Less.php
index dbda66cc948f1..f4339b40f5be8 100644
--- a/vendor/magento/module-deploy/Package/Processor/PreProcessor/Less.php
+++ b/vendor/magento/module-deploy/Package/Processor/PreProcessor/Less.php
@@ -59,6 +59,11 @@ class Less implements ProcessorInterface
*/
private $map = [];
+ /**
+ * @var array
+ */
+ private $pFileCache = [];
+
/**
* Less constructor
*
@@ -132,6 +137,7 @@ private function hasOverrides(PackageFile $parentFile, Package $package)
$currentPackageFiles = array_merge($package->getFilesByType('less'), $package->getFilesByType('css'));
foreach ($currentPackageFiles as $file) {
+ $this->pFileCache = [];
if ($this->inParentFiles($file->getDeployedFileName(), $parentFile->getFileName(), $map)) {
return true;
}
@@ -154,6 +160,10 @@ private function inParentFiles($fileName, $parentFile, $map)
return true;
} else {
foreach ($map[$parentFile] as $pFile) {
+ if (in_array($pFile, $this->pFileCache)) {
+ continue;
+ }
+ $this->pFileCache[] = $pFile;
if ($this->inParentFiles($fileName, $pFile, $map)) {
return true;
}
diff --git a/dev/tests/integration/testsuite/Magento/Deploy/DeployTest.php b/dev/tests/integration/testsuite/Magento/Deploy/DeployTest.php
index 72c722e139259..48f82fdcee184 100644
--- a/dev/tests/integration/testsuite/Magento/Deploy/DeployTest.php
+++ b/dev/tests/integration/testsuite/Magento/Deploy/DeployTest.php
@@ -240,7 +240,7 @@ private function assertCssFromChildTheme($actualRootCssContent)
private function assertBundleSize($theme)
{
$expectedSize = $this->bundleConfig->getBundleFileMaxSize('frontend', $theme);
- $expectedSize *= 1.15;
+ $expectedSize *= 1.2;
$iterator = $this->getDirectoryIterator("frontend/{$theme}/en_US/js/bundle");
/** @var \SplFileInfo $file */
Applying the changes in this patch to the Commerce 2.4.5 and 2.4.5-p1 installations drops the compilation time back from 80s to 28s
Hi @hostep @DuckThom,
Thank you for the response.
✔️ Issue confirmed
Issue got reproduce,After upgrading magento2.4.4 to magento 2.4.5.
Description:
After upgrading magento 2.4.4 to magento 2.4.5, while running setup:static-content:deploy command It is taking more time than magento 2.4.4
Pre-requisite:
- Magento 2.4.5
- PHP 8.1
Steps to reproduce:
-
Install Magento 2.4.4 instance and execute rm -R var/view_preprocessed/* pub/static/* && time php -d memory_limit=-1 bin/magento setup:static-content:deploy -f -j 4 en_US en_GB nl_NL nl_BE sv_SE command.
-
Upgrade Magento 2.4.4 to magento 2.4.5 and execute rm -R var/view_preprocessed/* pub/static/* && time php -d memory_limit=-1 bin/magento setup:static-content:deploy -f -j 4 en_US en_GB nl_NL nl_BE sv_SE command.
-
Compare both execution time.
Expected result: Execution time is same as Magento 2.4.4
Actual result: Execution time is more than magento 2.4.4
Screenshots:
1.In Magento 2.4.4
- Upgrade Magento 2.4.4 to Magento 2.4.5
- Magento 2.4.4 took 42s and Magento 2.4.5 took 51s .
Hence,confirming the issue.
Thanks
@engcom-Delta: can you verify it again on the latest 2.4-develop branch?
Like @DuckThom said, it sounds like it already got fixed in ACP2E-1168: Static content deployment timed out or failed
@hostep @DuckThom,
Yes, I agree with you. This issue is already got fixed in magento 2.4-develop. I have verified this in 2.4-develop
Hence I am closing this issue. Thanks.