Feature: Training tracking support with multiple loggers (i.e. Tensorboard, wandb)

[mayankjobanputra]

Haystack doesn't support any kind of training tracking today. The users who train the models can benefit if they can see the training progress and visualize some useful information (i.e. training/val loss). There're many different trackers like Tensorboard, wandb, etc. which can be supported but requires some structured planning.

Relevant discussion: https://github.com/deepset-ai/haystack/discussions/3414

[sjrl]

Hey, @mayankjobanputra just to let you know we do support some tracking through MLFlow which was added in this PR https://github.com/deepset-ai/haystack/pull/2337. There is some mention of MLFlow in our docs here (for Pipeline eval) and here for capturing some metrics during FARMReader training.

I do think we could benefit with some better docs around this tracking with MLFlow.

[masci]

@mayankjobanputra is this issue still relevant? In case it is, can we reformulate it to make it more actionable in light of @sjrl comments?

[mayankjobanputra]

imo it is relevant, but I will discuss it with you offline!

[SnoozingSimian]

Any updates on this? I would like something like this for my work. Is there a way I could help?

[nhadziosma1]

I agree, it would be really cool if there was a possibility to track the training and fine-tuning of models.