nni
nni copied to clipboard
Tensorboard logging
Describe the issue:
Hello, I'm using NNI v3.0 and trying to log using tensorboard. On the code I create the logging directory as log_dir = os.path.join(os.environ["NNI_OUTPUT_DIR"], "tensorboard")
but when I try to open tensorboard on the webgui doesn't appear any logging. Looking at the output directory I noticed that the NNI_OUTPUT_DIR points towards to
nni-experiments/dh1ou4w7/environments/local-env/trials/dqOLE/tensorboard
but tensorboard is trying to read
nni-experiments/dh1ou4w7/trials/QlXHm/output/tensorboard
How can I get the correct one and what's the difference of both?
Hello, is there any news regarding this?
I got same problem. I guess it is occurred from training service(local).
I fix the problem by the code below,
main .py code
log_dir = os.path.join(os.environ["PWD"], 'nni-experiments', os.environ["NNI_EXP_ID"], 'trials', os.environ["NNI_TRIAL_JOB_ID"], 'output/tensorboard')
.yml file
experimentWorkingDirectory: ./nni-experiments
I think it can be a problem when experiment setting is changed. I hope to know safe solution or the bug fixed.
Thanks for every contributors! NNI is really good and useful.