flow-development-collection icon indicating copy to clipboard operation
flow-development-collection copied to clipboard

TASK: Use `SCAN` for redis flush

Open kdambekalns opened this issue 1 year ago • 4 comments

This should speed up flush operations and lower the load on Redis.

Review instructions

The functional tests run flush()

Checklist

  • [x] Code follows the PSR-2 coding style
  • [x] Tests have been created, run and adjusted as needed
  • [x] The PR is created against the lowest maintained branch
  • [x] Reviewer - PR Title is brief but complete and starts with FEATURE|TASK|BUGFIX
  • [x] Reviewer - The first section explains the change briefly for change-logs
  • [x] Reviewer - Breaking Changes are marked with !!! and have upgrade-instructions

kdambekalns avatar Aug 29 '24 11:08 kdambekalns

Thx sound good! But what does should speed up exactly mean in numbers or examples? What was your use case?

I would like to test this thoroughly with some millions of tags in my projects.

Sebobo avatar Sep 09 '24 07:09 Sebobo

We saw high load on a Redis server caused by cache flushes calling KEYS. The fact that KEYS ist warned about in the documentation made me try this, and so far we saw a lower load. I don't have hard numbers, sadly, as I do not have direct access to the metrics…

kdambekalns avatar Sep 09 '24 07:09 kdambekalns

Just to confirm and avoid confusion: This method is only triggered manually in Production correct? I'm currently trying to come up with a "real" test case for the projects I know.

Sebobo avatar Sep 16 '24 08:09 Sebobo

Just to confirm and avoid confusion: This method is only triggered manually in Production correct? I'm currently trying to come up with a "real" test case for the projects I know.

In the project this caused issues, we see regular calls to that method. We still need to find out where they come from, to be honest. 🙈

That method is called (only) in AbstractFrontend->flush() and that is called automatically only in Develoment context … with the exception of node publishing, which triggers flush() on Flow_Persistence_Doctrine. Unless something custom in the project causes this, which is probable.

kdambekalns avatar Sep 16 '24 09:09 kdambekalns