Trace Files aren't written out when using dyno gputrace

[adirajagopal]

I'm currently trying to run inference profiling on a cuda kernel that's launched from pytorch. I am inside a docker container based which has cuda 12.5. I run:

• dynolog --enable_ipc_monitor --enable_gpu_monitor --dcgm_lib_path=/usr/lib/x86_64-linux-gnu/libdcgm.so

• KINETO_DAEMON_INIT_DELAY_S=3 KINETO_USE_DAEMON=True python profiling/profile.py : which runs the kernel I would like to profile in a loop of 100 iterations.

• The kernel being profiled is wrapped in the following torch profiler command: with torch.profiler.profiler(activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA], record_shapes=True, profile_memory=True, with_stack=True)

• When I launch the script, I get the message: INFO:2024-07-25 20:57:39 4278:4278 init.cpp:131] Registering daemon config loader, cpuOnly = 0

• I then run dyno gputrace --log-file /home/trace.json which outputs Trace output files will be written to: /home/trace_7605.json

But I never end up seeing the trace files that are written. I am using the latest version of dynolog, i.e. dynolog_0.3.2-0-amd64.deb. Any help on how I could get these traces would be much appreciated.