gramineproject
[LibOS,Pal] Support `/sys/devices/system/node/nodeX/meminfo` files
Description of the problem
One (private, not open source) workload fails because it doesn't find the following NUMA node info files:
[P1:T1:app] trace: ---- shim_stat("/sys/devices/system/node/node1/meminfo", 0x502768a96dc0) = -2
[P1:T1:app] trace: ---- shim_stat("/sys/devices/system/node/node0/meminfo", 0x502768a96dc0) = -2
Gramine currently doesn't emulate these files. We have emulation for a similar file /proc/meminfo, maybe we can re-use it here? But need to check.
Anyway, I think it should be trivial to add some (dummy) support for these files.
@vijaydhanraj Do you think it's doable and pretty simple to implement? Not right now of course, but in some near future.
yes, this should be simple to emulate, and we can reuse /proc/meminfo. I can do this after our 1.1 release.
We met similar issues when enabling a workload using libnuma. cc @RodgerZhu @xhuan28
This was planned for post-1.1, but as we've already been trending v1.3 release now, any blockers here to prevent this from being moved forward? And more generally, do we have any plan to enable libnuma? Thanks!
@kailun-qin I don't think there are any blockers. We simply didn't have time to work on this. If you (or someone else) can pick it up and implement quickly -- maybe after some discussions if needed -- then please go ahead. I can reassign this task from Vijay to someone else.
Got it, thanks! Sure, if @vijaydhanraj is fine w/ it, please kindly assign this to me. Myself (or my team) can help look into this.
Drafted a PR to add trivial support (similar to /proc/meminfo as suggested) for these files - just not sure whether the vaules here are still reasonable: https://github.com/gramineproject/gramine/blob/6c24e6a0932edfe84908e186ded5e3b622c0ddeb/libos/src/fs/sys/node_info.c#L129-L132
Normally the stats in /proc/meminfo would be the sum of those in /sys/devices/system/node/nodeX/meminfo, but it seems hard for Gramine SGX to emulate (though for Gramine direct we may do better by introducing some new PAL APIs, like this commit: https://github.com/gramineproject/gramine/pull/847/commits/e09846281b0fac4e304f6cf96e17927a8e51c762 does).
Pls kindly let me know if any better idea. Thanks!
@kailun-qin Let's continue the discussion in your PR #847. Generally, I don't like your current approach, see my comments in that PR.