Spark-PMoF icon indicating copy to clipboard operation
Spark-PMoF copied to clipboard

Published 20 hours ago • verified publisher icon Intel-bigdata

RDMA Enabling cannot allocate memory

Open kelvin-qin opened this issue 4 years ago • 1 comments

ENV:

  • Spark 2.3.1(hdp)
  • Hadoop 3.1.0
  • HiBench Terasort workload 500G data
  • RDMA nic CX4 ( or CX3 Pro) When I started rdma, I encountered an error,it said that fi_mr_reg: cannot allocate memory, and subsequently caused a NPE exception. When using the CX4 NIC, there is also an ArrayIndexOutOfBoundsException. The rping test has passed. The early configuration of RDMA is very complicated, is there any simpler solution to enable PMoF with rdma?

kelvin-qin avatar Aug 21 '20 06:08 kelvin-qin

The AEP has passed the test, and PMoF runs normally when the rdma function is not enabled.What kind of memory is the memory that cannot be allocated?Is it DRAM or devdax?

kelvin-qin avatar Aug 21 '20 06:08 kelvin-qin