Huge performance drop since 2.3.2 release

[jmsche]

Hello,

For the main project at work, I noticed a significant performance drop when upgrading from version 2.3.1 to either 2.3.2 or 2.3.4 (tested both).

At the beginning I suspected PHPUnit v12 upgrade (as I upgraded both at the same time), but after reverting everything & upgrading Foundry only, I figured out it is the culprit 😅

So about the performance drop:

• with Foundry 2.3.1, tests last around 1min 15sec
• with Foundry 2.3.2/2.3.4, tests last around 2min 15sec

Our test suite is half functional tests & half unit tests (2136 tests in total).

I'm going to try to investigate further, but rather tomorrow I think.

[nikophil]

I just tried to check the differences on a big testsuite (+4000 tests, 2/3 are functional tests) and I could not see a real difference 🤔

nor on the Foundry's testsuite 🤷

I'd be curious to know where it is coming from... Sadly, even if it was a bugfix release, we released a ton of stuff in v2.3.2

that won't be so easy to find where it is coming from... maybe this ?

What about the memory consumption?

[jmsche]

Hi nikophil,

Memory consumption is ~213MB with Foundry 2.3.1, and ~251MB with Foundry 2.3.4.

Using 2.3.4, I reverted the modifications in the PR you mentioned but this didn't change anything, I'll investigate this today & tomorrow.

[jmsche]

I did some vendor bisect (thanks Grégoire for their post), and if I did it correctly, here is the commit that impacts performance: https://github.com/zenstruck/foundry/commit/98f018c24c239b949654ba85e71cb54c0122ade9

Please tell me if I can help debug the issue (other than sharing the entire work project 😅)

[nikophil]

Memory consumption is ~213MB with Foundry 2.3.1, and ~251MB with Foundry 2.3.4.

ok, even if the memory is a little bit impacted, this is definitely not the culprit...

Using 2.3.4, I reverted the modifications in the PR you mentioned but this didn't change anything,

thanks, to have tried, at least :)

I did some vendor bisect (thanks Grégoire for their post), and if I did it correctly, here is the commit that impacts performance: https://github.com/zenstruck/foundry/commit/98f018c24c239b949654ba85e71cb54c0122ade9

super cool, thanks, this already really restricts the scope!

Please tell me if I can help debug the issue

Not sure what to do... In this PR we're calling $em->persist() on every entity created with Foundry. This allows to not depend on whether the relation is defined with cascade: ['persist'] or not. I'm quite surprised that the problem is coming from there, because in a way or another, at the end, all entities created by Foundry get added to the UoW 🤔

[nikophil]

hey, few more questions please:

• do you use dama?
• do you use Foundry's extension for PHPUnit?
• which version of Symfony & doctrine do you use?
• do you reset your database using ResetDatabase trait? which strategy?

few more things to try:

• can you try to run you test while commenting out this line?
• and another run, with these lines commented?

thanks 👋

I tried few things on Foundry's testsuite, but I cannot reproduce any overhead with the given PR

[jmsche]

Hi Nicolas,

• Yes, DAMA is used in the project
• Not using Foundry PHPUnit extension
• Symfony 7.2.3
• Doctrine:
  • DBAL: 3.9.4
  • ORM: 3.3.2
• Not using the ResetDatabase trait
• Tried commenting out the lines you mentioned, but no performance changes :/

[nikophil]

thanks tor these information.

Tried commenting out the lines you mentioned, but no performance changes

ok, I'm kinda relieved it's not coming from here 😅

[mdeboer]

I did notice a slight performance drop with 2.3.2, but updating to 2.3.10 helped. Haven't run any benchmarks to compare pre-2.3.2 to 2.3.10 but as performance was not really a problem for us, we didn't dig into it that much.

[nikophil]

any feedback would be welcome :)

nice to hear that 2.3.10 is better, hopefully it will be the last of the 2.3 series :)

note that Foundry is currently known to perform too much flushes (REALLY too much 😅 ), and I have some work in progress to fix this. This might impact the perf in a good way :)

[jmsche]

Hi @jasperpoppe, as you are still on the impacted project, could you please check if latest version of Foundry is okay? Thanks :)

[jasperpoppe]

Package is now on 2.3.10. Tests: 2192, Time: 02:06.646, Memory: 241.50 MB

[nikophil]

Nothing spectacular, then 🤷‍♂️

Thanks for the input

[mdeboer]

@jasperpoppe can you share some insights on the duration per test?

Is there a global increase in time for all tests or are there specific tests which took a performance hit?

If there are specific tests, can you share some information about them what makes them different from test that took no/less of a performance hit?

[nikophil]

I'm closing this issue which is stale for more than 3 months. Feel free to reopen it to provide more information that could help to find where the performance drop was coming from