Training metrics

[louisfd]

In burn-train, several metrics can be used during training. It would be great to have more!

• [X] Accuracy
• [X] Loss (the one in use)
• [X] CUDA utilization (memory&compute)
• [x] Top-k accuracy
• [x] CPU utilization
• [x] CPU memory usage
• [ ] General GPU utilization
• [ ] General GPU memory usage
• [ ] Precision Recall
• [ ] AUC - ROC
• [ ] BLEU score
• [ ] ROUGE score

[Elazrod56]

Hello, can I be assigned ? I'll deal with it

[louisfd]

Of course! 😄

[Elazrod56]

If I understood correctly, all Rust files that get metrics are located in burn-train/src/metric/

Just to check, is my top_k_acc.rs file in the right place ?

[louisfd]

Yes, right there :)

[Elazrod56]

Just a question, by "CPU Memory usage", do you mean the RAM or something else ?

[nathanielsimard]

Just the RAM, nothing fancy :)

[Elazrod56]

I can't get myself to figure out how non-hardware metrics work, what they mean and how to implement them. Is that okay if I only implement the hardware metrics (CPU use, memory use, GPU use, GPU memory) ? I'll add temperature metrics for the CPU and the GPU to compensate my incapability to understand deep learning-specific metrics.

[nathanielsimard]

Yes no problem, you can submit a PR with only one metric if you want!

[oleksii-shyman]

Is this issue still relevant? I am interested to continue unfinished metrics.

I have the following questions:

1. I have AMD Radeon RX 5500 GPU on my machine. Is it good enough?
2. I am not really familiar with deep learning metrics, but I am willing to understand them. Would experienced dev be able to guide me in case I have questions?

[nathanielsimard]

You can add metrics with any machine. To understand them, you can probably read some articles first like https://neptune.ai/blog/performance-metrics-in-machine-learning-complete-guide and then maybe use wikipedia for each individual metric you want to add.