rust-executors
rust-executors copied to clipboard
Bathtor
NUMA Aware Work Stealing
When running with pinned threads, prioritise NUMA local work-stealing (significantly) over stealing from further away cores.
Status
- [x] Partially available with "manual" specification of a PU distance matrix (from #10 )
- [ ] Automatic discovery is blocked on better hwloc support or some topology alternative
C&P from Slack for reference:
lkroll 14:52 Alright, I’ve been looking into the NUMA aware workstealing and it’s a bit tricky. The only way I found to get some sensible topology information is via rust-hwloc(2), which is a FFI wrapper around libhwloc. So far so good, but the people who wrote the wrapper clearly have no idea how Rust FFIs are supposed to work and their interfaces suck balls. In particular some of their methods throw SIGSEGV because they try to dereference NULL pointers within an unsafe block (which isn’t exposed…the function itself is written as if it were safe). 14:52 Now, we could try to fix this…but that would probably mean we’ll end up maintaining rust-hwloc in long term (at least that is my experience with these kind of semi-abandoned projects) 14:53 And when I say ‘we’, I obviously mean ‘you’, since I’ll be leaving after December :D 14:53 How would you like me to proceed on this?^^ New Max 14:56 I see, well it isn't critical to have right now anyways. More of a wish. Thanks for checking, I might have a look into it again in the future :smile: