DAOS-15824 vos: per-pool based pre-allocated DTX LRU array

[Nasf-Fan]

The DTX LRU array is pre-allocated in DRAM for active DTX entries. Currently, it is per-container based. In theory, for each opened container, the count of such pre-allocated DTX entries can be up to 1^20 at most. Each DTX entry will occupy 264 bytes. Then the DRAM usage will be 264MB per-container shard. From server perspective, under the worst case, the total DRAM usage will be:

264MB * targets_per_engine * engines_per_server

For large system with a lot of pools and containers, that will be terrible.

This patch changes the DTX LRU array from container to pool. If there are a lot of containers (assume the count as N) within the same pool, then it can reduce the server DRAM consumption to 1 / N for the best case, that is much helpful.

Before requesting gatekeeper:

• [ ] Two review approvals and any prior change requests have been resolved.
• [ ] Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
• [ ] Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
• [ ] Commit messages follows the guidelines outlined here.
• [ ] Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

• [ ] You are the appropriate gatekeeper to be landing the patch.
• [ ] The PR has 2 reviews by people familiar with the code, including appropriate owners.
• [ ] Githooks were used. If not, request that user install them and check copyright dates.
• [ ] Checkpatch issues are resolved. Pay particular attention to ones that will show up on future PRs.
• [ ] All builds have passed. Check non-required builds for any new compiler warnings.
• [ ] Sufficient testing is done. Check feature pragmas and test tags and that tests skipped for the ticket are run and now pass with the changes.
• [ ] If applicable, the PR has addressed any potential version compatibility issues.
• [ ] Check the target branch. If it is master branch, should the PR go to a feature branch? If it is a release branch, does it have merge approval in the JIRA ticket.
• [ ] Extra checks if forced landing is requested
  • [ ] Review comments are sufficiently resolved, particularly by prior reviewers that requested changes.
  • [ ] No new NLT or valgrind warnings. Check the classic view.
  • [ ] Quick-build or Quick-functional is not used.
• [ ] Fix the commit message upon landing. Check the standard here. Edit it to create a single commit. If necessary, ask submitter for a new summary.

[github-actions[bot]]

Ticket title is 'Per-pool based pre-allocated DTX LRU array ' Status is 'In Review' Labels: 'md_on_ssd2' https://daosio.atlassian.net/browse/DAOS-15824

[jolivier23]

The only concern I have is that when you reload a container, the stored LID for a transaction needs to be available. Is there any case where that won't be possible? Or is there something that explicitly avoids this possibility.

[Nasf-Fan]

The only concern I have is that when you reload a container, the stored LID for a transaction needs to be available. Is there any case where that won't be possible? Or is there something that explicitly avoids this possibility.

That is a good question. I will consider more. Thanks!

[Nasf-Fan]

The only concern I have is that when you reload a container, the stored LID for a transaction needs to be available. Is there any case where that won't be possible? Or is there something that explicitly avoids this possibility.

That is a good question. I will consider more. Thanks!

Related issue has been resolved in the latest commit, please review again. Thanks!

[Nasf-Fan]

Ping reviewers, thanks!

[Nasf-Fan]

Ping reviewers, thanks!

[Nasf-Fan]

The main issue to be resolved via the patch is for the case of sparse pre-allocated active DTX array may occupy a lot of DRAM. Under very bad situation, a opened container may occupy at most 264MB DRAM. If there are a lot of opened containers, GB level DRAM may be occupied. But of course, it is for some extreme cases. On the other hand, there are some short-comings if changed to pre-pool based active DTX table, for example, one container trouble (holding a lot of active DTX entires for quite long time) may affect the other containers. So we need more consideration about whether it is worth to make such adjustment.