pmdk
pmdk copied to clipboard
Published
20 hours ago •
pmem
pmem
Create obj on /dev/daxX.Y with no badblock failed on checking badblock
On Arm64 ubuntu 21.04. The /dev/dax0.0 is the Device-dax.
I think there maybe some issue here to check the badblock? Using the ndctl, for dax0.0 device, there is no badblocks shows, but I'm not sure why the libpmemobj try to checking the badblock.
sudo ndctl list
[
{
"dev":"namespace1.0",
"mode":"fsdax",
"map":"dev",
"size":8453619712,
"uuid":"47e5123c-1452-4c06-a2ed-138f8380c701",
"sector_size":512,
"align":2097152,
"blockdev":"pmem1"
},
{
"dev":"namespace0.0",
"mode":"devdax",
"map":"dev",
"size":4225761280,
"uuid":"f73af9e7-59e7-4363-abdd-972d5b2921aa",
"chardev":"dax0.0",
"align":4096
}
]
sudo ndctl list --dimms --regions --health --media-errors --human
{
"dimms":[
{
"dev":"nmem1",
"id":"8680-57341200",
"handle":"0x2",
"phys_id":"0"
},
{
"dev":"nmem0",
"id":"8680-56341200",
"handle":"0x1",
"phys_id":"0"
}
],
"regions":[
{
"dev":"region1",
"size":"8.00 GiB (8.59 GB)",
"align":"16.00 MiB (16.78 MB)",
"available_size":0,
"max_available_extent":0,
"type":"pmem",
"mappings":[
{
"dimm":"nmem1",
"offset":"0",
"length":"0x200000000",
"position":0
}
],
"persistence_domain":"unknown"
},
{
"dev":"region0",
"size":"4.00 GiB (4.29 GB)",
"align":"16.00 MiB (16.78 MB)",
"available_size":0,
"max_available_extent":0,
"type":"pmem",
"mappings":[
{
"dimm":"nmem0",
"offset":"0",
"length":"0x100000000",
"position":0
}
],
"persistence_domain":"unknown"
}
]
}
Log as below:
MEMOBJ_LOG_LEVEL=15 ./pmempool create obj /home/ubuntu/pmempool.testset1
<libpmemobj>: <1> [out.c:205 out_init] pid 20768: program: /home/ubuntu/pmdk/src/tools/pmempool/pmempool
<libpmemobj>: <1> [out.c:208 out_init] libpmemobj version 2.4
<libpmemobj>: <1> [out.c:212 out_init] src version: 1.11.0+git8.ge72106c45
<libpmemobj>: <1> [out.c:220 out_init] compiled with support for Valgrind pmemcheck
<libpmemobj>: <1> [out.c:225 out_init] compiled with support for Valgrind helgrind
<libpmemobj>: <1> [out.c:230 out_init] compiled with support for Valgrind memcheck
<libpmemobj>: <1> [out.c:235 out_init] compiled with support for Valgrind drd
<libpmemobj>: <1> [out.c:240 out_init] compiled with support for shutdown state
<libpmemobj>: <1> [out.c:245 out_init] compiled with libndctl 63+
<libpmemobj>: <3> [mmap.c:39 util_mmap_init]
<libpmemobj>: <3> [libpmemobj.c:23 libpmemobj_init]
<libpmemobj>: <3> [obj.c:256 obj_init]
<libpmemobj>: <3> [obj.c:156 obj_ctl_init_and_load] pop (nil)
<libpmemobj>: <3> [set.c:125 util_remote_init]
<libpmemobj>: <3> [obj.c:1319 pmemobj_createU] path /home/ubuntu/pmempool.testset1 layout (null) poolsize 0 mode 664
<libpmemobj>: <3> [obj.c:1290 obj_get_nlanes]
<libpmemobj>: <3> [set.c:3352 util_pool_create] setp 0xffffd3852c00 path /home/ubuntu/pmempool.testset1 poolsize 0 minsize 8388608 minpartsize 2097152 attr 0xffffd3852c60 nlanes 0xffffd3852bf4 can_have_rep 1
<libpmemobj>: <3> [set.c:3111 util_pool_create_uuids] setp 0xffffd3852c00 path /home/ubuntu/pmempool.testset1 poolsize 0 minsize 8388608 minpartsize 2097152 pattr 0xffffd3852c60 nlanes 0xffffd3852bf4 can_have_rep 1 remote 0
<libpmemobj>: <3> [file.c:37 util_file_exists] path "/home/ubuntu/pmempool.testset1"
<libpmemobj>: <3> [set.c:2123 util_poolset_create_set] setp 0xffffd3852c00 path /home/ubuntu/pmempool.testset1 poolsize 0 minsize 8388608
<libpmemobj>: <3> [file.c:115 util_file_get_type] path "/home/ubuntu/pmempool.testset1"
<libpmemobj>: <3> [file.c:37 util_file_exists] path "/home/ubuntu/pmempool.testset1"
<libpmemobj>: <3> [file.c:481 util_file_open] path "/home/ubuntu/pmempool.testset1" size 0xffffd3852a50 minsize 0 flags 0
<libpmemobj>: <3> [file.c:170 util_fd_get_size] fd 3
<libpmemobj>: <3> [source_posix.c:92 pmem2_source_size] type 2
<libpmemobj>: <4> [source_posix.c:132 pmem2_source_size] file length 29
<libpmemobj>: <4> [file.c:199 util_fd_get_size] file length 29
<libpmemobj>: <4> [file.c:522 util_file_open] actual file size 29
<libpmemobj>: <3> [set.c:1485 util_poolset_parse] setp 0xffffd3852c00 path /home/ubuntu/pmempool.testset1 fd 3
<libpmemobj>: <10> [set.c:1543 util_poolset_parse] PMEMPOOLSET
<libpmemobj>: <3> [set.c:1093 util_parse_add_replica] setp 0xffffd38529a8
<libpmemobj>: <10> [set.c:767 parser_read_line] size 'AUTO' path '/dev/dax0.0'
<libpmemobj>: <3> [file_posix.c:36 util_is_absolute_path] path: /dev/dax0.0
<libpmemobj>: <3> [file.c:115 util_file_get_type] path "/dev/dax0.0"
<libpmemobj>: <3> [file.c:37 util_file_exists] path "/dev/dax0.0"
<libpmemobj>: <4> [pmem2_utils_linux.c:52 pmem2_get_type_from_stat] device subsystem path "/sys/dev/char/249:3/subsystem"
<libpmemobj>: <3> [file.c:150 util_file_get_size] path "/dev/dax0.0"
<libpmemobj>: <3> [file.c:170 util_fd_get_size] fd 5
<libpmemobj>: <4> [pmem2_utils_linux.c:52 pmem2_get_type_from_stat] device subsystem path "/sys/dev/char/249:3/subsystem"
<libpmemobj>: <3> [source_posix.c:92 pmem2_source_size] type 2
<libpmemobj>: <3> [region_namespace_ndctl.c:136 pmem2_region_namespace] ctx 0xaaaaeba30580 src 0xaaaaeba26400 pregion (nil) pnamespace 0xffffd3852748
<libpmemobj>: <3> [region_namespace_ndctl.c:28 ndctl_match_devdax] st_rdev 63747 devname dax0.0
<libpmemobj>: <4> [region_namespace_ndctl.c:51 ndctl_match_devdax] found matching device: /dev/dax0.0
<libpmemobj>: <4> [pmem2_utils_ndctl.c:88 pmem2_device_dax_size] device size 4225761280
<libpmemobj>: <4> [source_posix.c:132 pmem2_source_size] file length 4225761280
<libpmemobj>: <4> [file.c:199 util_fd_get_size] file length 4225761280
<libpmemobj>: <3> [set.c:1073 util_parse_add_element] set 0xaaaaeba26370 path /dev/dax0.0 filesize 4225761280
<libpmemobj>: <3> [set.c:991 util_parse_add_part] set 0xaaaaeba26370 path /dev/dax0.0 filesize 4225761280
<libpmemobj>: <3> [set.c:979 util_replica_add_part] replica 0xaaaaeba252e0 path "/dev/dax0.0" filesize 4225761280
<libpmemobj>: <3> [set.c:932 util_replica_add_part_by_idx] replica 0xaaaaeba252e0 path /dev/dax0.0 filesize 4225761280
<libpmemobj>: <3> [set.c:902 util_replica_reserve] replica 0xaaaaeba252e0 n 1
<libpmemobj>: <3> [file.c:115 util_file_get_type] path "/dev/dax0.0"
<libpmemobj>: <3> [file.c:37 util_file_exists] path "/dev/dax0.0"
<libpmemobj>: <4> [pmem2_utils_linux.c:52 pmem2_get_type_from_stat] device subsystem path "/sys/dev/char/249:3/subsystem"
<libpmemobj>: <3> [file_posix.c:143 util_file_device_dax_alignment] path "/dev/dax0.0"
<libpmemobj>: <3> [file_posix.c:111 device_dax_alignment] path "/dev/dax0.0"
<libpmemobj>: <4> [pmem2_utils_linux.c:52 pmem2_get_type_from_stat] device subsystem path "/sys/dev/char/249:3/subsystem"
<libpmemobj>: <3> [region_namespace_ndctl.c:136 pmem2_region_namespace] ctx 0xaaaaeba28900 src 0xaaaaeba26f40 pregion (nil) pnamespace 0xffffd3852760
<libpmemobj>: <3> [region_namespace_ndctl.c:28 ndctl_match_devdax] st_rdev 63747 devname dax0.0
<libpmemobj>: <4> [region_namespace_ndctl.c:51 ndctl_match_devdax] found matching device: /dev/dax0.0
<libpmemobj>: <4> [pmem2_utils_ndctl.c:48 pmem2_device_dax_alignment] device alignment 4096
<libpmemobj>: <3> [set.c:1166 util_poolset_check_devdax] set 0xaaaaeba26370
<libpmemobj>: <3> [set.c:1391 util_poolset_directories_load] set 0xaaaaeba26370
<libpmemobj>: <4> [set.c:1674 util_poolset_parse] set file format correct (/home/ubuntu/pmempool.testset1)
<libpmemobj>: <3> [set.c:1203 util_poolset_check_options] set 0xaaaaeba26370
<libpmemobj>: <3> [set.c:1219 util_poolset_set_size] set 0xaaaaeba26370
<libpmemobj>: <3> [set.c:1255 util_poolset_set_size] pool size set to 4225761280
<libpmemobj>: <3> [set_badblocks.c:81 badblocks_check_poolset] set 0xaaaaeba26370 create 1
<libpmemobj>: <3> [set.c:4245 util_poolset_foreach_part_struct] set 0xaaaaeba26370 callback 0xffffa6d94cfc arg 0xffffd3852a60
<libpmemobj>: <3> [set_badblocks.c:34 badblocks_check_file_cb] part_file 0xffffd3852a20 arg 0xffffd3852a60
<libpmemobj>: <3> [file.c:37 util_file_exists] path "/dev/dax0.0"
<libpmemobj>: <3> [bad_blocks.c:250 badblocks_check_file] file /dev/dax0.0
<libpmemobj>: <3> [bad_blocks.c:24 badblocks_count] file /dev/dax0.0
<libpmemobj>: <3> [badblocks.c:18 badblocks_new]
<libpmemobj>: <3> [bad_blocks.c:47 badblocks_get] file /dev/dax0.0 badblocks 0xaaaaeba26550
<libpmemobj>: <4> [pmem2_utils_linux.c:52 pmem2_get_type_from_stat] device subsystem path "/sys/dev/char/249:3/subsystem"
<libpmemobj>: <3> [badblocks_ndctl.c:253 pmem2_badblock_context_new] src 0xaaaaeba325c0 bbctx 0xffffd38528f0
<libpmemobj>: <3> [region_namespace_ndctl.c:136 pmem2_region_namespace] ctx 0xaaaaeba2dc50 src 0xaaaaeba325c0 pregion 0xffffd3852860 pnamespace 0xffffd3852868
<libpmemobj>: <3> [region_namespace_ndctl.c:28 ndctl_match_devdax] st_rdev 63747 devname dax0.0
<libpmemobj>: <4> [region_namespace_ndctl.c:51 ndctl_match_devdax] found matching device: /dev/dax0.0
<libpmemobj>: <3> [badblocks_ndctl.c:106 badblocks_get_namespace_bounds] region 0xaaaaeba2e010 namespace 0xaaaaeba346f0 ns_offset 0xffffd3852870 ns_size 0xffffd3852878
<libpmemobj>: <1> [badblocks_ndctl.c:133 badblocks_get_namespace_bounds] (dax) cannot read offset of the namespace
<libpmemobj>: <1> [badblocks_ndctl.c:313 pmem2_badblock_context_new] cannot read namespace's bounds
<libpmemobj>: <3> [extent_linux.c:155 pmem2_extents_destroy] extents 0xaaaaeba3d130
<libpmemobj>: <3> [badblocks.c:34 badblocks_delete] badblocks 0xaaaaeba26550
<libpmemobj>: <1> [bad_blocks.c:254 badblocks_check_file] counting bad blocks failed -- '/dev/dax0.0'
<libpmemobj>: <1> [set_badblocks.c:56 badblocks_check_file_cb] checking the pool file for bad blocks failed -- '/dev/dax0.0'
<libpmemobj>: <1> [set.c:3182 util_pool_create_uuids] failed to check pool set for bad blocks -- '/home/ubuntu/pmempool.testset1'
<libpmemobj>: <3> [set.c:522 util_poolset_free] set 0xaaaaeba26370
<libpmemobj>: <2> [obj.c:1354 pmemobj_createU] cannot create pool or pool set
error: '/home/ubuntu/pmempool.testset1' -- checking the pool file for bad blocks failed -- '/dev/dax0.0'
error: creating pool file failed
<libpmemobj>: <3> [libpmemobj.c:36 libpmemobj_fini]
<libpmemobj>: <3> [obj.c:300 obj_fini]
<libpmemobj>: <3> [set.c:140 util_remote_fini]
<libpmemobj>: <3> [set.c:195 util_remote_unload]
<libpmemobj>: <3> [mmap.c:73 util_mmap_fini]
ISSUE:
Environment Information
- PMDK package version(s):
- OS(es) version(s):
- ndctl version(s):
- kernel version(s):
- compiler, libraries, packaging and other related tools version(s):
Please provide a reproduction of the bug:
How often bug is revealed: (always, often, rare):
Actual behavior:
Expected behavior:
Details
Additional information about Priority and Help Requested:
Are you willing to submit a pull request with a proposed change? (Yes, No)
Requested priority: (Showstopper, High, Medium, Low)
More info, I can get the "resource" and "size" from:
root@kevin-ubuntu-21:/sys/dev/char/249:3/subsystem/dax0.0# ls
align dev device mapping0 modalias numa_node power resource size subsystem target_node uevent
root@kevin-ubuntu-21:/sys/dev/char/249:3/subsystem/dax0.0# cat resource
0x244200000
root@kevin-ubuntu-21:/sys/dev/char/249:3/subsystem/dax0.0# cat size
4225761280
root@kevin-ubuntu-21:/sys/dev/char/249:3/subsystem/dax0.0#
libpmemobj checks bad blocks and shutdown state by default. AFAIR one of the problems of the existing ndctl interface is that there's no way to distinguish between "bad blocks not supported" and "retrieving bad blocks failed", and so we resort to just failing - which is the safer thing to do, but can produce false errors as you've noticed. The recommended thing to do on platforms that don't support bad blocks is manually disabling this feature: https://pmem.io/pmdk/manpages/linux/master/pmempool/pmempool-feature.1.html
Nevertheless, I think the way this works right now can be annoying, so I'll see if we can look into improving the situation.
