[Benchmarking] Add portable machine and benchmark context detection
The machine detection script (make_machine_json.sh) currently looks like it will only work properly on macOS. Ideally it should work more or less correctly on all of them macOS, Linux and Windows (some values may remain undetected on some platforms).
This probably entails:
- extend
archery benchmark(see https://jira.apache.org/jira/browse/ARROW-4827 and tentative implementation in https://github.com/apache/arrow/pull/4077/files) rather than a standalone bash script - using something like psutil to grab useful machine information
- calling
nvidia-smito query GPU characteristics (for example "nvidia-smi -q -i 0 -x")
Reporter: Antoine Pitrou / @pitrou
Related issues:
Note: This issue was originally created as ARROW-4716. Please see the migration documentation for further details.
Antoine Pitrou / @pitrou: cc @tanyaschlusser
Tanya Schlusser / @tanyaschlusser: (y)
Francois Saint-Jacques / @fsaintjacques: Do we need this to be cross-platform, I think Linux should suffice first. It might be worth writing this in python for future compat with without.
Tanya Schlusser / @tanyaschlusser: Linux first is a great choice.
Python is my favorite language and I am happy to do this. The existing shell script was only because it was easy and quick.
I am very very sorry for dropping the ball the past couple of months. My mom passed away and I have been a total wreck (and moved back home from her house and got a job) but I still want to contribute and hope you will accept me back now. You all rock very much!
Francois Saint-Jacques / @fsaintjacques: No need to be sorry! Hope you feel better. I'll rename and re-organize this ticket.
This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.