daos icon indicating copy to clipboard operation
daos copied to clipboard

Published 20 hours ago • verified publisher icon daos-stack

DAOS-11043 common: Blob backed Memory(BMEM) interface for umem

Open sherintg opened this issue 2 years ago • 18 comments

This commit contains a new allocator which is a modified version of pmemobj. The new allocator operates on volatile memory and provides transaction functionality similar to that of pmemobj.

The BMEM interfaces internally use the new allocator functionalties to implement the required backend for hosting metadata on volatile memory backed by blob.

Note: The current commit default to BMEM instead of PMDK. One can override this by setting DAOS_USE_PMDK=1 in the server config file.

Allow-unstable-test: true

Signed-off-by: sherintg [email protected]

Before requesting gatekeeper:

  • [ ] Two review approvals and any prior change requests have been resolved.
  • [ ] Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
  • [ ] Commit messages follows the guidelines outlined here.
  • [ ] Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

  • [ ] You are the appropriate gatekeeper to be landing the patch.
  • [ ] The PR has 2 reviews by people familiar with the code.
  • [ ] Any appropriate watchers have had a chance to review the PR.
  • [ ] Review comments are sufficiently resolved, particularly by prior reviewers that requested changes.
  • [ ] Githooks were used. If not, there is a sufficient reason to move forward. Check copyrights if githooks were not used.
  • [ ] Checkpatch issues are resolved. Pay particular attention to ones that will show up on future PRs.
  • [ ] All builds have passed. Check non-required builds to ensure they don't indicate any problem such as a compiler warning on extraneous platforms.
  • [ ] No new NLT or valgrind warnings. Check the classic view. This step should only matter if the build requires force landing.
  • [ ] Ensure sufficent testing is done. Check feature pragmas and test tags. Check that tests skipped for the ticket are run and now pass with the changes.
  • [ ] Quick-build or Quick-functional is not used.
  • [ ] Pay attention to PRs that may affect compatibility between versions and ensure it has been addressed.
  • [ ] Check the target branch. If it is master branch, should the PR go to a feature branch? If it is a release branch, does it have merge approval in the JIRA ticket.
  • [ ] Check the commit message when landing. Check the standard here. Edit it to create a single commit. If necessary, ask submitter for a new summary.

sherintg avatar Oct 10 '22 07:10 sherintg

Bug-tracker data: Ticket title is 'Implement new alternative umem backend' Status is 'In Progress' Labels: 'md_on_ssd' Job should run at elevated priority (3) https://daosio.atlassian.net/browse/DAOS-11043

github-actions[bot] avatar Oct 10 '22 08:10 github-actions[bot]

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/2/execution/node/145/log

daosbuild1 avatar Oct 10 '22 09:10 daosbuild1

Test stage NLT on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/2/execution/node/638/log

daosbuild1 avatar Oct 10 '22 10:10 daosbuild1

Test stage Unit Test on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/2/execution/node/620/log

daosbuild1 avatar Oct 10 '22 10:10 daosbuild1

Test stage Unit Test with memcheck on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/2/execution/node/618/log

daosbuild1 avatar Oct 10 '22 10:10 daosbuild1

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/3/execution/node/145/log

daosbuild1 avatar Oct 10 '22 13:10 daosbuild1

Test stage Unit Test with memcheck on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/3/execution/node/563/log

daosbuild1 avatar Oct 10 '22 14:10 daosbuild1

Test stage Unit Test on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/3/execution/node/598/log

daosbuild1 avatar Oct 10 '22 14:10 daosbuild1

Test stage NLT on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/3/execution/node/638/log

daosbuild1 avatar Oct 10 '22 14:10 daosbuild1

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/4/execution/node/145/log

daosbuild1 avatar Oct 11 '22 05:10 daosbuild1

Test stage NLT on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/4/execution/node/624/log

daosbuild1 avatar Oct 11 '22 06:10 daosbuild1

Bug-tracker data: Ticket title is 'Implement new alternative umem backend' Status is 'In Progress' Labels: 'md_on_ssd' Job should run at elevated priority (3) Errors are Unknown component https://daosio.atlassian.net/browse/DAOS-11043

It doesn't like umem. You can either change it to another directory name under src/ or src/client or add umem as a component in VALID_COMPONENTS in ci/jira_query.py

This is not required but if you push again, this will eliminate the noise.

jolivier23 avatar Oct 11 '22 20:10 jolivier23

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/5/execution/node/145/log

daosbuild1 avatar Oct 12 '22 11:10 daosbuild1

Test stage NLT on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/5/execution/node/638/log

daosbuild1 avatar Oct 12 '22 11:10 daosbuild1

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/7/execution/node/146/log

daosbuild1 avatar Oct 13 '22 03:10 daosbuild1

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/7/execution/node/1091/log

daosbuild1 avatar Oct 13 '22 10:10 daosbuild1

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/8/execution/node/145/log

daosbuild1 avatar Oct 14 '22 14:10 daosbuild1

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/8/execution/node/1090/log

daosbuild1 avatar Oct 15 '22 03:10 daosbuild1

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/9/execution/node/144/log

daosbuild1 avatar Oct 19 '22 12:10 daosbuild1

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/9/execution/node/1083/log

daosbuild1 avatar Oct 20 '22 07:10 daosbuild1

I merged latest base branch and tested the failing core tests. I wasn't able to reproduce the rebuild failure

jolivier23 avatar Oct 21 '22 00:10 jolivier23

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/10/execution/node/145/log

daosbuild1 avatar Oct 25 '22 16:10 daosbuild1

Test stage Functional on EL 8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-10525/10/testReport/(root)/

daosbuild1 avatar Oct 25 '22 22:10 daosbuild1

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-10525/11/execution/node/145/log

daosbuild1 avatar Oct 26 '22 21:10 daosbuild1

Test stage Functional on EL 8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-10525/11/testReport/(root)/

daosbuild1 avatar Oct 27 '22 06:10 daosbuild1