Callback for imagehistory (for semantic segmentation)

[andife]

🚀 Feature

Progress tracking during a semantic segmentation task. Visualization of the prediction of the current state of the model on custom-defined example images. (Maybe some obvious classes and more difficult ones with specific characteristics.

Motivation

I would to I just started using aim and the first impression is really good. Now I'm trying it to integrate it into my current workflow. This is mainly semantic segmentation with tensorflow/keras. The changelog mentions that at version 1.1.13/1.2.14/1.2.16. But I didn't see an example for the up to now. Maybe I haven't found the part in docs so far. For tensorboard, I've written a custom callback function for that purpose.

Pitch

One possible way could to be to visualize three selected output classes as an RGB image, with one class at the different channels. Besides, I think it is helpful to do this on a specific set of pre-selected images.

Additional context

[alberttorosyan]

Hey @andife! Thanks for opening this issue. Below is the basic aim callback for keras: https://github.com/aimhubio/aim/blob/27d978072606122a7c0b84c4fdff3cc8a70b0dfd/aim/sdk/adapters/keras.py#L10 We can use this as a starting point for more advanced callback implementation. Also it would be great if you can share the tensorbard custom callback code here for more context.

[andife]

imagehistory.txt

and I use the callback with;

v_data = [next(myGen_demo) for item in range(3)]
print("range: " + str(range(3)))
print("type vdata: " + str(type(v_data)))
print(str(len(v_data)))
image_history = ImageHistory(
    tensor_board_dir=tensor_board_directory,
    data=v_data,
    last_step=2,
    draw_interval=100,
    selectslices=[0, 1, 2]
)

This callback "image_history" I use at the model.fit stage