noobaa-core icon indicating copy to clipboard operation
noobaa-core copied to clipboard

πŸŒΈπŸ’ mdsequence cherries

Open baum opened this issue 2 years ago β€’ 0 comments

PostgreSQL-related performance optimization.

Those changes were πŸŒΈπŸ’ cherry-picked from dannyzaken/noobaa-core/tree/danny-mdsequence

  • Use PostgreSQL sequence
    • currently mdsequence is using a counter inside a jsonb field, which has very bad performance in parallel writes.
    • changed to use postgres native SEQUENCE.
  • Increase PostgreSQL client connections pool size if LOCAL_MD_SERVER
    • for the endpoints, we saw that when a large number of small writes is done in parallel, a lot of queries are waiting for clients. the DB has max connections of 600, so it should be able to handle more connections.
  • Remove sort by version_enabled in find_object_null_version
    • For some reason, the generated query is causing Postgres to perform a sort according to version_enabled, even though the index is also on version_enabled.
    • version_enabled is always null in this case, and the sort is redundant. removing it solved the issue.

baum avatar Jul 27 '22 13:07 baum