Redis / AOF - Magento 2.4.6-p3

[Nuranto]

Summary

With redis for session, configured with appendonly=yes (AOF), we noticed that after upgrading from 2.4.6-p2 to 2.4.6-p3, storage was increasing very very fast (More than 5Go used for 300Mo of actual data). Our redis servers were crashing because of that (volume getting full within 24hours...). We guess the session are updated more frequently than before.

A hotfix was to switch to RDB.

I'm not sure it is related to 2.4.6-p3, could be another composer dependency upgrade. But I wanted to let you know, maybe others have experienced this issue too ?

Examples

Proposed solution

No response

Release note

No response

Triage and priority

• [ ] Severity: S0 - Affects critical data or functionality and leaves users without workaround.
• [X] Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
• [ ] Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
• [ ] Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
• [ ] Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

[m2-assistant[bot]]

Hi @Nuranto. Thank you for your report. To speed up processing of this issue, make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, Add a comment to the issue:

• @magento give me 2.4-develop instance - upcoming 2.4.x release
• For more details, review the Magento Contributor Assistant documentation.
• Add a comment to assign the issue: @magento I am working on this
• To learn more about issue processing workflow, refer to the Code Contributions.

---

Join Magento Community Engineering Slack and ask your questions in #github channel. :warning: According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting. :clock10: You can find the schedule on the Magento Community Calendar page. :telephone_receiver: The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.

[Nuranto]

Update : We still have issues with RDB. So the issue could be the number of sessions that increased, or the size of them. We'll continue investigate.

[ananth-iyer]

@Nuranto it could be related to Magento CSP: Please check: https://github.com/magento/magento2/issues/29964

[Nuranto]

I'm not sure, this ticket seems related to cache. We only have issues with sessions.

[hostep]

Not many session related changes at first sight: https://github.com/magento/magento2/compare/2.4.6-p2..2.4.6-p3

Maybe one of the recent changes in these 3 repositories causes it?

• https://github.com/colinmollenhour/php-redis-session-abstract
• https://github.com/colinmollenhour/Cm_Cache_Backend_Redis
• https://github.com/colinmollenhour/credis

[Nuranto]

Yes, that was my thought too. But you're right, redis dependencies have been updated at the same time, I'll look into it, thanks.

[Garde85]

I have a similar issue. After upgrade Magento from 2.3.5 to 2.4.6-p2, Redis memory start to grow very very fast and arrived to 11 GB. In my case memory saturation seems not to be related to session, because I use 2 different Redis instances: one for sessions and one for caches. Instance that manage sessions does not have any problems, while instance that manage backend cache saturate his memory. I observer that the isseu seems to be related to 2 type of magento cache:

• layout
• block_html

if i flush theese caches, memory size of redis return to (around) 300MB.

[m2-assistant[bot]]

Hi @engcom-November. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
• [ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue.
• [ ] 3. Add Area: XXXXX label to the ticket, indicating the functional areas it may be related to.
• [ ] 4. Verify that the issue is reproducible on 2.4-develop branch
  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
• [ ] 5. Add label Issue: Confirmed once verification is complete.
• [ ] 6. Make sure that automatic system confirms that report has been added to the backlog.

[engcom-November]

Hello @Nuranto,

Thank you for the report and collaboration!

We configured redis for session with readonly, and upgraded from 2.4.6-p2 to the latest 2.4.7-beta2. But we didn't see any significant increase in redis size. Please take a look at the screenshot: This is the reddis size with 2.4.6-p2

And this is when upgraded to 2.4.7-beta2

Please let us know if we are missing anything.

Thank you.

[JoostWan]

Same problem over here will multiple shops and Redis page cache.

[goivvy]

@Nuranto Were you able to fix it? What version of redis are you running, was it upgraded along with Magento?

[amcguireweb]

We're currently updating to 2.4.6-p3 in hopes to solve our Redis issues, this does not sound promising. Has anyone determined the issue yet?

[Nuranto]

Hi @goivvy, no we did not upgrade redis along with magento. We're running on redis 7.0.14. And we stabilized this by disabling AOF

[hostep]

I just noticed this commit made in core magento that limits 2 of the redis packages to a certain version (according to the commit message - in broken English - it sounds like newer versions of these packages have performance issues): https://github.com/magento/magento2/commit/9369884f8c40745291347470c5910a0f01a556af

[engcom-November]

@Nuranto, @hostep we were not able to reproduce this issue on vanilla 2.4-develop instance with redis AOF turned on. Are there any additional things required in order to reproduce this issue, for example should we have a large number of data in magento instance or large number of traffic is required? could you please provide more information on this.

Thank you.

[amcguireweb]

@Nuranto, @hostep we were not able to reproduce this issue on vanilla 2.4-develop instance with redis AOF turned on. Are there any additional things required in order to reproduce this issue, for example should we have a large number of data in magento instance or large number of traffic is required? could you please provide more information on this.

Thank you.

We're all experiencing the issue. It's not new to the most recent version, it's been a problem through all past versions. I am guessing you installed vanilla magento, with no data, and no traffic? You'll need a large catalog and simulated traffic to reproduce this issue.

[kestraly]

Currently experiencing this issue on 3 different setups. I've switched over to RDB, which has indeed solved much of the issue. But I noticed Redis continued to spike. All 3 versions are on Magento 2.4.5 and all 3 versions had "Cache User Defined Attributes" - YES. Cache EAV types and attributes - whilst enabled seems to grow without limitation. Whilst disabled, the Redis database stabilises