HIP icon indicating copy to clipboard operation
HIP copied to clipboard

Clarification of hipMallocManaged memory placement (GPU or CPU on Frontier/Crusher)

Open bgoglin opened this issue 1 year ago • 2 comments

Hello I was reading the OLCF documentation of Frontier/Crusher systems and got confused by section "HSA_XNACK=0 Automatic Page Migration Disabled" on page https://docs.olcf.ornl.gov/systems/crusher_quick_start_guide.html#notable-differences-between-summit-and-crusher Second line of the table says that hipMallocManaged() initially allocates on CPU DDR4, and then CPU accesses are "Zero copy read/write over Infinity Fabric" while GPU accesses are "Local read/write". Wait, migration is disabled in this case so memory won't ever move to GPU HBM. Hence GPU access shouldn't be "local read/write" but rather "Zero copy read/write over Infinity Fabric" (and the contrary of CPU access). Or is the memory initially allocated on GPU when using hipMallocManaged() ? Thanks Brice

bgoglin avatar Mar 30 '23 07:03 bgoglin