Memory Region Failure ibv_reg_mr failed [error code: 14]

[agnesnatasya]

Hi,

Problem

When setting up, I encountered ibv_reg_mr failed [error code: 14] error, indicating that the memory region requested is a bad / invalid address. Here are the values that I have

• At libfs/lib/global/global.h
  • g_log_size 32768UL // (128 MB)
• At libfs/lib/rdma/mr.c
  • mrs[0] = {addr = 140733004644352, length = 4299161600}
  • mrs[1] = {addr = 140728713871360, length = 111071232}
• At vi /etc/default/grub
  • GRUB_CMDLINE_LINUX="memmap=8G!4G"
• Size configuration
  • python3 ./utils/change_dev_size.py 8 0 0
• At /etc/security/limits.conf

  •  * soft memlock unlimited
     * hard memlock unlimited
    

Do you mind helping me out with this problem?

Thank you!

[wreda]

This likely indicates that the memory you're trying to register is out-of-bounds.

Try setting your device size to a value lower than 8G using the change_dev_size.py script. After doing so, recompile LibFS/KernFS, and then rerun mkfs.sh.

[agnesnatasya]

You're correct, thank you very much for the suggestion and help!

Do you mind if I clarify why the problem occured? Assise's NVM region is 8GiB and from what I understand, the actual size of the emulated NVM is indeed 8GiB, as shown through ndctl list

{
  "dev":"namespace0.0",
  "mode":"fsdax",
  "map":"mem",
  "size":8589934592,
  "sector_size":512,
  "blockdev":"pmem0",
  "numa_node":0
}

[wreda]

I believe this is due to an alignment issue. I'll leave this open for now and will check later if there's a possible fix for this.