Allow ad-hoc color changing

[martinpopel]

As discussed in #288 and #449, it is difficult to choose one color palette that suits everyone in all situations. If there are many runs and I select just some of them (with a regex or toggling individual runs), I quite often end up with e.g., red, red, orange, red and I cannot distinguish the runs especially if the curves are crossing each other. I suggest to add a drop-down color picker (e.g. using http://jscolor.com/) to choose a color for each run. I plan to use this especially before posting TensorBoard screenshots to my colleagues.

[nfelt]

This could be useful. Note that jscolor itself though we couldn't use since it's GPL.

[martinpopel]

OK, jscolor is not needed (it was just an example). Choosing e.g. from 16 pre-defined colors would be enough for my needs.

[jart]

Our team probably can't do something like this at the moment. I suspect configuring color might be more appropriate if done in code/config rather than UI.

[martinpopel]

I understand there may be higher priority features to implement. Just note that configuring colors in code/config would not solve this issue. The point is that I don't know in advance which runs I will want to compare. I want to change the colors ad hoc from the GUI. Also in case of multiple users connecting to the same TensorBoard server, I think the colors should be per-user configurable, i.e. client-side.

It is a question whether to solve the run<-->color mapping (semi)permanently in the browser. If possible this may be useful. Another issue with TensorBoard is that when I delete a run (e.g. to save disk space), all colors change, so I have to redo all screenshots again to have consistent colors.

[netheril96]

A way to change color in config/code/command line is better than nothing, although I'd favor a Web UI too.

[jart]

Is the root problem that colors are not predictable and might change as you add additional runs? I'm not sure how color selection works at the moment. Although one possible thing to consider is using hash(run_name) % #colors to make color selection somewhat more deterministic.

Could that solve your problem?

[martinpopel]

The color of a given run changes only if I delete any older runs. This is bothersome, but it is not the main issue I originally reported.

My issue is: I have hundreds of runs in TensorBoard and I select various subsets using regex filters or checkboxes. Each subset has at most eight runs, so it would be possible to use distinct color for each run, but quite often this is not the case.

Maybe there could be a button "change colors" which would automatically select colors so the currently selected runs have distinct colors - this would solve my issue. However, I think my original proposal is more useful: by manual ad-hoc coloring, the users can choose the colors exactly as they want (e.g. using red for a "failed" run and gray for baseline). Of course, we could have both a button for automatic re-coloring and an ad-hoc color picker.

BTW: My current workaround is to create symlinks for a given run until one of the symlinks gets assigned a suitable color. Then I keep visible just that symlink and hide all previous symlinks plus the original.

[jart]

We're currently building SQL support so Runs can be grouped into Experiments, Experiments can be grouped into Projects, which are owned by Users. The next logical step is to add a UI feature that lets you compare an intersection of experiments.

In other words, we're working towards better solutions to organizing data. Do you suspect that might solve the root problem here?

[martinpopel]

All that is great, but I don't see how it could help with this problem without adding the color picker. I keep my runs organized in a directory structure already, e.g. training-dataset/number-of-gpus/batch-size/learning-rate/warmup-steps/etc. In one minute I want to see all runs on a given training data, then I want to compare runs with all hyperparmeters fixed except for batch size, then I want to see the effect of number of GPUs, etc.. So there is no single way how to organize the runs in Projects/Experiments/Groups such that I would always compare only runs within one group. Specifically for this issue, there is no fixed mapping of runs to colors which would suit all my needs.

[jart]

One thing you might be able to do is render a matplotlib image and log it as an image summary.

[martinpopel]

One thing you might be able to do is render a matplotlib image and log it as an image summary.

This is not even a workaround (and it would require hack all the TF frameworks again): I don't know in advance which colors should each run have - I decide this only when using TensorBoard and selecting the subset of runs.

My current workaround is to export the plots to csv files and plot these with gnuplot-lua-tikz (which is more suitable for academic papers than screenshots anyway). But this is not suitable for interactive analysis in TensorBoard (zooming, exact numbers it tooltips, etc.), similarly as what you suggested.

I think this discussion starts getting too long and not focused on the main problem and its solution (color picker), but rather on workarounds. If you feel this issue has low-priority and there is no one to implement it, let's close it and go on. A summary of workarounds (for other users):

• clicking the show/hide checkbox for each run there and back to see "which of the four red curves (crossing each other) is which one"
• changing the color of a given run using the symlink hack
• create a new directory with symlinks to the runs you are interested in and start a new TensorBoard instance with --logdir pointing to this directory
• exporting the runs to csv and re-plotting in your favorite visualization tool

[netheril96]

I have the same needs as the author of this issue. The emphasis is on ad-hoc, that is, the fact that our selection is constantly changing on the spot. None of the workarounds proposed here work that dynamically. They are all fixed beforehand.

[djl11]

I agree with both netheril96 and martinpopel. Making sense of data which are marked with similar colors is difficult, particularly when 20+ scalars are being plotted on a single graph, and colors are re-used. This feature request would be a great addition to tensorboard.

[jart]

What do we think about the following solutions?

1. Have a checkbox next to regex search so it chooses new colors each time it's applied.
2. Make (1) the default behavior and don't bother with the checkbox.
3. Make (2) cleverer, by only choosing new colors if the filtered colors needlessly overlap.

[martinpopel]

@jart: These three solutions may help and as someone here wrote, anything is better than nothing, but...

The problem of 2 and 3 is that some colors will change when users don't expect this - I guess this could be very controversial (although for me it is better than the current state). Sometimes I get used to some color-run mappings. Sometimes I make the regex more specific just temporarily then want to go back and see the same colors as before. The problem of all three solutions is that users don't have full control over the colors, so it will solve just some use cases. Sometimes I need to display 20+ runs and even if you automatically choose a different color for each, there will be some difficult to distinguish pairs and no automatic algorithm can guess which runs are important for me to distinguish. Another real-life example: I have 5 settings evaluated on two test sets, resulting in 10 runs (curves) in one graph. One test set is easier, so none of its accuracy curves cross the other test set curves. As a visual aid I would like to mark the same setting in both test sets with the same color, thus using only 5 colors for the 10 runs.

So my suggestion is: 4. Add a drop-down color picker for each run. It should be easy to implement and solves all the problems very elegantly. There could be a icon for the color picker next to the check-box and radio-button, or it could be activated by left-click or right-click on the run name, so it does not take any extra space. 5. In addition to 4, there could be a button which assigns different colors for all the runs in the current regex filter randomly. So you can click on it again and again until you are satisfied. This could be helpful if someone is lazy to choose a color for each run manually (e.g. because changing the regex filter frequently, but I guess this is not so common use case). 6. In addition to 4 and 5, when a run is deleted from disk, no colors should be changed.

Note that solution 4 is what I am suggesting all the time from my first post and I still don't see what's the problem with this approach. I am afraid explaining why it is better than all the suggested alternatives took me more effort than actually implementing it and sending a PR.

[djl11]

I agree that custom color selection would be by far the best solution. As Martin popel said, it would even be useful to colour certain plots the same color sometimes.

For example: 3 datasets, each with 3 hyperparameter selections, and 3 different architectures, making 27 scalars plotted in total. Maybe I could use just red blue green to distinguish the dataset for 1 screenshot, then the hyperparameters for another, and finally the architectures.

This is a simple example, but custom color selection would really add a great deal of visualisation power I feel, which is of course what tensorboard is all about, and otherwise does so well at.

But anything is better than nothing of course, I’m not sure how difficult this is to implement. I very much appreciate your attention on this.

[djl11]

Also, as a side note. Most people in my lab use tensorboard regularly, and by far the most common use case is comparing many runs in a single plot, with various parameters selected for cross comparison, with directory structure for the log files used to distinguish meaning. Having briefly spoken to each of them today after writing this comment, this is a feature request which all of them were very keen for. Of course this is not a very formal poll! But I genuinely think it is a feature which would be very much welcomed by the research community who use tensorboard.

[kocmitom]

I would also greatly appreciated the possibility to change the colors ad-hoc. Or just one button "reassign colors", which would take into account the runs I selected first (I usually select much less than 16 runs).

[OverLordGoldDragon]

Any updates? TF2's out, TB remains unicolored - the issue's still relevant

[jrenrut]

Hi, are there any updates on this issue? I'm having the same problem but with the scatter plot colors with the tensorboard projector.

[diggerdu]

What about developing a mode for colorblind users and a mode for normal people, that would meet the needs of everyone

[nicofirst1]

Any update on this?

[ham952]

i want to present the same problem with a different perspective. I did a (single) training itteration with multiple "stops" , analysing the parameters and then re-continuing the same. Such that by the end of complete run , i had many sessions and ofcourse each session plot is showing a different color .. Below is the snapshot of same ..

Now i have compare it with another iteration, completed in a simialr fashion .. for me the problem is two-fold .. With so many color combinations its really frustrating to even segregate the two main itterations .. for me the solution can be a directory (NOT sub-directory) based color assignment .. i-e all the sub-directory in a main directory can have a single color

-iteration01 (red)

• run01
• run02
• run03

-iteration02 (blue)

• run01
• run02
• run03

I beleive many people facing a similar problem or even in a different setting, can find such a solution more helpful.

[martinpopel]

@ham952: Why don't you move the event files from all sub-directories to the same directory? This can be done with a single mv command (or you can configure your tools to store the even files there in the first place). The event files should have unique filenames and TensorBoard should merge the datapoints from all the files automatically into a single run (there may be problems with overlapping curves, ie. two different y-values for the same x-value, but it seems to happen just rarely in your case).

Note also that while using automatically the same color for all sub-directories would make you happy, it would make almost everyone sad. What could work (in addition to my suggestions 4,5,6 above) is an extra color-picker which assigns the same color to all currently selected runs. This way you could regex-select all the runs which should be red, then all the runs which should be blue etc. This global color-picker could be near the button which assigns a random different colors to all selected runs (which was my suggestion 5).

[ham952]

@ham952: Why don't you move the event files from all sub-directories to the same directory? This can be done with a single mv command (or you can configure your tools to store the even files there in the first place). The event files should have unique filenames and TensorBoard should merge the datapoints from all the files automatically into a single run (there may be problems with overlapping curves, ie. two different y-values for the same x-value, but it seems to happen just rarely in your case).

Note also that while using automatically the same color for all sub-directories would make you happy, it would make almost everyone sad. What could work (in addition to my suggestions 4,5,6 above) is an extra color-picker which assigns the same color to all currently selected runs. This way you could regex-select all the runs which should be red, then all the runs which should be blue etc. This global color-picker could be near the button which assigns a random different colors to all selected runs (which was my suggestion 5).

Thanks @martinpopel .. it solved my particular problem .. didnt had the idea that tensorboard can merge the datapoints from multiple event files ..

Stay Blessed !

[dgrahn]

As an alternative to a full-blown UI solution, could we have the option to load a new palette of colors? It would be nice to have... 20 different colors rather than the few there are now.

[Trezorro]

Any update on this? Some of the UI based suggestions really seem like low hanging fruit for improving Tensorboard significantly.

It may be a good option to implement it as a color parameter for tf.summary.create_file_writer(). This would already open up a world of possibilities, and may be easier than UI changes.

Just to reiterate, UI options would always be better in dealing with the ad-hoc needs of a user during analysis.

[jkyl]

As an alternative to a full-blown UI solution, could we have the option to load a new palette of colors? It would be nice to have... 20 different colors rather than the few there are now.

or at least not gray... can i get an amen

[fanyuzeng]

how to change the curve color of tensorboard?

[dgrahn]

Alrighty. I got tired of this lingering, so I decided to take it on. I can't change the assignee, but if someone could assign me, that would be great.

I don't have a full solution yet, but I can get you 20 colors. Working on making this a toggleable option now.

• Download the source for TensorBoard
• Follow the instructions to run it.
• Swap out tensorboard/components/tf_color_scale/palettes.ts with the code from this Gist: https://gist.github.com/dgrahn/ac4a567ee023cd7d326602d4f15ee71d