Can tensorboard support loading event logs from HDFS?

[cheyang]

I found tensorboard is an awesome tool, and also used it to analyze the events generated by tensorflow. Now I'd like to analyze the events in the HDFS. Can we support this? Thanks.

[zihaolucky]

Thanks for the idea, but what's the use case? Does it mean we could remote logging and rendering?

The event file is relatively small as we haven't support graph visualization and embedding, so it might be useful if the event file becomes super large when we support embedding later.

[cheyang]

The scenario is for tf.contrib.learn which is the high level API. In this case, when we set model_dir as "hdfs://namenode:9000/data", the event logs are stored in HDFS which we need to run tensorboard to load.

m = tf.contrib.learn.DNNClassifier(model_dir=model_dir,
                                       feature_columns=columns['data'],
                                       hidden_units=HIDDEN_UNITS)

[zihaolucky]

Interesting, let me do some research on this. The NDArray of MXNet could also be stored in s3 or HDFS, may be a good material for me to start.

[jimdowling]

We would love to be able to run Tensorboard with logs in HDFS. Currently, we have to copy logs from hdfs to local disk and then run Tensorboard.