[4.x]: (large) site becomes unusable due to generating project config YAML files

[engram-design]

What happened?

I've noticed an issue with project config for some time now, and it's really noticeable for large sites. In this context, this is for one of the test sites we (Verbb) use for development, so we really put Craft through its paces with lots of fields, sections, entry types, and content rows - so it's a good test case. And with 50+ plugins of ours to test, there's lots of stuff we need!

Now, that might not be a real-world case, but it certainly brings up a noticeable issue, and that's generating the project config YAML files. When disabling every plugin, my local server will timeout trying to create these files. It's at the point where I have to disable the generation of the YAML files with:

'components' => [
    'projectConfig' => function() {
        $config = craft\helpers\App::projectConfigConfig();
        $config['writeYamlAutomatically'] = false;
        return Craft::createObject($config);
    },
],

You can see in this video that the process stalls without even getting to the end.

What's more, I noticed that even on a smaller site with less data, the project config is rebuilt multiple times. See this video. Here, I've extended the timeout to illustrate the issue, but even I run out of patience waiting for it to finish 😁

You'll notice I'm not even making any changes, just loading the control panel to trigger the YAML files to be generated. It's incredibly slow on its own - I'm not exactly sure why? Sure, disk I/O is an expensive process, but this seems to be very slow to generate files. Understandably, when generating it multiple times, things may fall over, but it can barely get through one round.

I'm using MAMP, and while - yes - there might be better options out there - I've found it generally much faster than Docker, Valet, etc. I don't think it's an issue with my local environment, but happy to be proven wrong.

I'm more than happy to send through the database and project files, as they are just local testing stuff. Just let me know! I'd firstly like to see if this is just an issue for me, expected behaviour or something you've also noticed.

Craft CMS version

4.6

PHP version

No response

Operating system and version

No response

Database type and version

No response

Image driver and version

No response

Installed plugins and versions

[brandonkelly]

You'll notice I'm not even making any changes, just loading the control panel to trigger the YAML files to be generated.

I can see that in your video – but they didn’t exist to begin with, so it’s expected that Craft would generate them in that case. (The only exception is if writeYamlAutomatically is disabled.) Once they’re generated, I’m assuming the files aren’t continuing to get regenerated when simply accessing the CP without doing anything?

The generation does sure seem to be especially slow, though.

Couple things to try:

• Disable Xdebug. Depending on the version and how it’s configured, that can cause PHP to run significantly slower. (You can verify it’s disabled from Utilities → PHP Info.)
• Disable Blackfire.
• Run composer dump-autoload -o from your project root.

If those don’t make a big dent, go ahead and send your Composer files and a database backup over to [email protected] and we can take a look.

[engram-design]

Totally understand Craft needs to generate them! But the same thing happens when I save a field, the entire project folder of YAML files is re-created in its entirety. You're correct that once generated, navigating around the CP is fine - I'm just trying to illustrate the generation of YAML files is slow.

Xdebug is disabled, running PHP 8.0.28 (but 8.2.0 is the same), and no Blackfire. The composer command does nothing, so sending those files over!

Seeing as though you think that's unusual, I'll do some more testing on my environment to see if that's the issue - it could very well be the case.

[brimby]

Just want to add that I have this exact issue as well so I'll be observing what happens here and can send files too if it comes down to that. This install is fairly old and I had hoped that upgrading to 4 would magically cure it, but no. I'm using MAMP currently but I'm going to move to a remote dev environment soon and will report back if the env change helped.