gpu-bdb
gpu-bdb copied to clipboard
WIP: log dask task stream and rmm events into results dir
Adds a couple of utilities to capture metrics per query triggered in benchmark_runner.py rmm_monitor logs Cuda allocations from each dask worker dask_task_logger grabs the tasks into a json file
TBD:
- gate with an environment variable as the logs add up quickly.
- mark rmm logs with worker ID
I have another commit pending which fetches the logs into a single df which can then be serialized (parquet?) Still need an example analysis. Probably go back to a bokeh plot that works as a standalone example.