xdp-tutorial
xdp-tutorial copied to clipboard
xdp-project
How to use 100000 hash arrays with map?
I want make a hash tree with a lot of hash arrays(may be 1000000). But one program can access at most 64 maps. I have tried to use a map with a value size =1000000*size(int), but failed when load the program. Or is there any way that the hash map can add a new element when update an existed element ？ who can help me ?
You can create a hash map with 1000000 slots in it. If you need each value to be an array, you just define the value size as your array size and interpret it as an array in your program. This should work, only limited by system memory...
when I set a map (hash type or no hash type array)with max entries of 1000000 and value size of sizeof(int test), the kernel program can load successfully. Or I define a array in kernel problem as follow: static int test[1000000]; ,then the program can also load successfully. All of the above tell us the system memory is enough. But when I set a map with max entries of 1 and value size of sizeof(int test[1000000]), then the program load failed， when value size reduced to sizeof(int test[5000]), the it can load successfully. So，I want to know what configuration limit the value size, and how to resolve it???
sofardware [email protected] writes:
when I set a map (hash type or no hash type array)with max entries of 1000000 and value size of sizeof(int test), the kernel program can load successfully. Or I define a array in kernel problem as follow: static int test[1000000]; ,then the program can also load successfully. All of the above tell us the system memory is enough. But when I set a map with max entries of 1 and value size of sizeof(int test[1000000]), then the program load failed， when value size reduced to sizeof(int test[5000]), the it can load successfully. So，I want to know what configuration limit the value size, and how to resolve it???
You can't have a value size higher than KMALLOC_MAX_SIZE, which is 4 MB. Why would you want to create such an array, though? Just create an array map with 1000000 elements in it?
I need 10000 different hash arrays (evry one have 100 slots) in my policy tree , and there may be same key with different value in two or more different hash arrays. If I split a map with 1000000 elements to 10000 arrays， the value in first array will be overwritten by the value in second array if they have the same key value. How can resolve this problem?
One method I have found is to split the key value = "array Index"(hi bits) + "hash value"(low bits) . Dose it ok ? Or if there is any better method？
I can't have a value size higher than KMALLOC_MAX_SIZE, which is 4 MB. My map is only 10000 bytes as follow. But it still failed, why ???: struct bpf_map_def SEC("maps") test = { .type = BPF_MAP_TYPE_ARRAY, .key_size = sizeof(int), .value_size = 10000, .max_entries = 1, };
root packet03-redirecting(master *%)$ ./xdp_loader -d eth2 -F -S --progsec xdp_router libbpf: load bpf program failed: Permission denied libbpf: -- BEGIN DUMP LOG --- libbpf: 0: (61) r6 = *(u32 *)(r1 +4) 1: (7b) *(u64 *)(r10 -184) = r1 2: (61) r8 = *(u32 *)(r1 +0) 3: (b7) r7 = 0 4: (7b) *(u64 *)(r10 -8) = r7 last_idx 4 first_idx 0 regs=80 stack=0 before 3: (b7) r7 = 0 5: (7b) *(u64 *)(r10 -16) = r7 6: (7b) *(u64 *)(r10 -24) = r7 7: (7b) *(u64 *)(r10 -32) = r7 8: (7b) *(u64 *)(r10 -40) = r7 9: (7b) *(u64 *)(r10 -48) = r7 10: (7b) *(u64 *)(r10 -56) = r7 11: (7b) *(u64 *)(r10 -64) = r7 12: (73) *(u8 *)(r10 -134) = r7 13: (b7) r1 = 2605 14: (6b) *(u16 *)(r10 -136) = r1 15: (18) r1 = 0x2d2d2d2d2d2d2d2d 17: (7b) *(u64 *)(r10 -144) = r1 18: (7b) *(u64 *)(r10 -152) = r1 19: (18) r1 = 0x2d2d2d2d2d2d2d3a 21: (7b) *(u64 *)(r10 -160) = r1 22: (18) r1 = 0x656e696765623a65 24: (7b) *(u64 *)(r10 -168) = r1 25: (18) r1 = 0x74756f7274736574 27: (7b) *(u64 *)(r10 -176) = r1 28: (bf) r1 = r10 29: (07) r1 += -176 30: (b7) r2 = 43 31: (85) call bpf_trace_printk#6 last_idx 31 first_idx 0 regs=4 stack=0 before 30: (b7) r2 = 43 32: (bf) r5 = r8 33: (bf) r9 = r5 34: (07) r9 += 14 35: (b7) r8 = 1 36: (2d) if r9 > r6 goto pc+482 R0_w=inv(id=0) R5_w=pkt(id=0,off=0,r=14,imm=0) R6_w=pkt_end(id=0,off=0,imm=0) R7_w=invP0 R8_w=inv1 R9_w=pkt(id=0,off=14,r=14,imm=0) R10=fp0 fp-8_w=00000000 fp-16_w=00000000 fp-24_w=00000000 fp-32_w=00000000 fp-40_w=00000000 fp-48_w=00000000 fp-56_w=00000000 fp-64_w=00000000 fp-136=?????mmm fp-144_w=mmmmmmmm fp-152_w=mmmmmmmm fp-160_w=mmmmmmmm fp-168_w=mmmmmmmm fp-176_w=mmmmmmmm fp-184_w=ctx 37: (71) r1 = *(u8 *)(r5 +12) 38: (71) r2 = *(u8 *)(r5 +13) 39: (67) r2 <<= 8 40: (4f) r2 |= r1 41: (55) if r2 != 0x8 goto pc+26 R0=inv(id=0) R1=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R2=inv8 R5=pkt(id=0,off=0,r=14,imm=0) R6=pkt_end(id=0,off=0,imm=0) R7=invP0 R8=inv1 R9=pkt(id=0,off=14,r=14,imm=0) R10=fp0 fp-8=00000000 fp-16=00000000 fp-24=00000000 fp-32=00000000 fp-40=00000000 fp-48=00000000 fp-56=00000000 fp-64=00000000 fp-136=?????mmm fp-144=mmmmmmmm fp-152=mmmmmmmm fp-160=mmmmmmmm fp-168=mmmmmmmm fp-176=mmmmmmmm fp-184=ctx 42: (bf) r1 = r5 43: (07) r1 += 34 44: (b7) r8 = 1 45: (2d) if r1 > r6 goto pc+473 R0=inv(id=0) R1_w=pkt(id=0,off=34,r=34,imm=0) R2=inv8 R5=pkt(id=0,off=0,r=34,imm=0) R6=pkt_end(id=0,off=0,imm=0) R7=invP0 R8_w=inv1 R9=pkt(id=0,off=14,r=34,imm=0) R10=fp0 fp-8=00000000 fp-16=00000000 fp-24=00000000 fp-32=00000000 fp-40=00000000 fp-48=00000000 fp-56=00000000 fp-64=00000000 fp-136=?????mmm fp-144=mmmmmmmm fp-152=mmmmmmmm fp-160=mmmmmmmm fp-168=mmmmmmmm fp-176=mmmmmmmm fp-184=ctx 46: (71) r1 = *(u8 *)(r5 +22) 47: (b7) r8 = 2 48: (2d) if r8 > r1 goto pc+470 R0=inv(id=0) R1_w=inv(id=0,umin_value=2,umax_value=255,var_off=(0x0; 0xff)) R2=inv8 R5=pkt(id=0,off=0,r=34,imm=0) R6=pkt_end(id=0,off=0,imm=0) R7=invP0 R8_w=inv2 R9=pkt(id=0,off=14,r=34,imm=0) R10=fp0 fp-8=00000000 fp-16=00000000 fp-24=00000000 fp-32=00000000 fp-40=00000000 fp-48=00000000 fp-56=00000000 fp-64=00000000 fp-136=?????mmm fp-144=mmmmmmmm fp-152=mmmmmmmm fp-160=mmmmmmmm fp-168=mmmmmmmm fp-176=mmmmmmmm fp-184=ctx 49: (7b) *(u64 *)(r10 -192) = r2 50: (b7) r1 = 2 51: (73) *(u8 *)(r10 -64) = r1 52: (71) r1 = *(u8 *)(r5 +15) 53: (73) *(u8 *)(r10 -52) = r1 54: (71) r1 = *(u8 *)(r5 +23) 55: (73) *(u8 *)(r10 -63) = r1 56: (b7) r1 = 0 57: (6b) *(u16 *)(r10 -62) = r1 last_idx 57 first_idx 41 regs=2 stack=0 before 56: (b7) r1 = 0 58: (6b) *(u16 *)(r10 -60) = r1 59: (69) r1 = *(u16 *)(r5 +16) 60: (dc) r1 = be16 r1 61: (6b) *(u16 *)(r10 -58) = r1 62: (61) r1 = *(u32 *)(r5 +26) 63: (63) *(u32 *)(r10 -48) = r1 64: (bf) r6 = r5 65: (61) r1 = *(u32 *)(r5 +30) 66: (63) *(u32 *)(r10 -32) = r1 67: (05) goto pc+47 115: (b7) r1 = 10 116: (6b) *(u16 *)(r10 -148) = r1 117: (b7) r1 = 1680162168 118: (63) *(u32 *)(r10 -152) = r1 119: (18) r1 = 0x65646e6966695f73 121: (7b) *(u64 *)(r10 -160) = r1 122: (18) r1 = 0x736572676e693a65 124: (7b) *(u64 *)(r10 -168) = r1 125: (18) r1 = 0x74756f7274736574 127: (7b) *(u64 *)(r10 -176) = r1 128: (79) r8 = *(u64 *)(r10 -184) 129: (61) r3 = *(u32 *)(r8 +12) 130: (bf) r1 = r10 131: (07) r1 += -176 132: (b7) r2 = 30 133: (85) call bpf_trace_printk#6 last_idx 133 first_idx 115 regs=4 stack=0 before 132: (b7) r2 = 30 134: (61) r1 = *(u32 *)(r8 +12) 135: (63) *(u32 *)(r10 -56) = r1 136: (bf) r2 = r10 137: (07) r2 += -64 138: (bf) r1 = r8 139: (b7) r3 = 64 140: (b7) r4 = 0 141: (85) call bpf_fib_lookup#69 last_idx 141 first_idx 115 regs=8 stack=0 before 140: (b7) r4 = 0 regs=8 stack=0 before 139: (b7) r3 = 64 142: (7b) *(u64 *)(r10 -200) = r0 143: (b7) r1 = 0 144: (b7) r2 = 0 145: (85) call bpf_map_lookup_elem#1 R1 type=inv expected=map_ptr processed 92 insns (limit 1000000) max_states_per_insn 0 total_states 3 peak_states 3 mark_read 2
libbpf: -- END LOG -- libbpf: failed to load program 'xdp_router' libbpf: failed to load object 'xdp_prog_kern.o' ERR: loading BPF-OBJ file(xdp_prog_kern.o) (-22): Invalid argument ERR: loading file: xdp_prog_kern.o
sofardware [email protected] writes:
I can't have a value size higher than KMALLOC_MAX_SIZE, which is 4 MB. My map is only 10000 bytes as follow. But it still failed, why ???: struct bpf_map_def SEC("maps") test = { .type = BPF_MAP_TYPE_ARRAY, .key_size = sizeof(int), .value_size = 10000, .max_entries = 1, };
root packet03-redirecting(master *%)$ ./xdp_loader -d eth2 -F -S --progsec xdp_router libbpf: load bpf program failed: Permission denied
This looks like a ulimit error...
sofardware [email protected] writes:
I need 10000 different hash arrays (evry one have 100 slots) in my policy tree , and there may be same key with different value in two or more different hash arrays. If I split a map with 1000000 elements to 10000 arrays， the value in first array will be overwritten by the value in second array if they have the same key value. How can resolve this problem?
You can put maps inside maps. Look at these two map types:
BPF_MAP_TYPE_ARRAY_OF_MAPS,
BPF_MAP_TYPE_HASH_OF_MAPS,
sofardware [email protected] writes: I can't have a value size higher than KMALLOC_MAX_SIZE, which is 4 MB. My map is only 10000 bytes as follow. But it still failed, why ???: struct bpf_map_def SEC("maps") test = { .type = BPF_MAP_TYPE_ARRAY, .key_size = sizeof(int), .value_size = 10000, .max_entries = 1, }; root packet03-redirecting(master *%)$ ./xdp_loader -d eth2 -F -S --progsec xdp_router libbpf: load bpf program failed: Permission denied This looks like a ulimit error...
I have run ulimit -l unlimited. Is any other parameter of ulimit need to set ？
sofardware [email protected] writes:
sofardware [email protected] writes: I can't have a value size higher than KMALLOC_MAX_SIZE, which is 4 MB. My map is only 10000 bytes as follow. But it still failed, why ???: struct bpf_map_def SEC("maps") test = { .type = BPF_MAP_TYPE_ARRAY, .key_size = sizeof(int), .value_size = 10000, .max_entries = 1, }; root packet03-redirecting(master *%)$ ./xdp_loader -d eth2 -F -S --progsec xdp_router libbpf: load bpf program failed: Permission denied This looks like a ulimit error...
I have run ulimit -l unlimited. Is any other parameter of ulimit need to set ？
No, ulimit -l is enough. Since you're getting the verifier dump, I guess this is not a ulimit issue after all (my bad, should have realised that from the start), but rather the verifier is rejecting your program. Probably because you're missing a bounds check somewhere...