online hard-example mining under Multi-GPU (DDP)

[bhack]

🚀 The feature

Do you think it would be possible to have an off the shelf component like the one described at https://github.com/Lightning-AI/pytorch-lightning/discussions/1170?

Motivation, pitch

Sometime on unbalanced dataset it could be interesting to make specific sampling related to perplexity or othermetrics and loss values. It could be also interesting for implementing some type of online curriculum learning/sampling solution.

Alternatives

No response

Additional context

Write and maintain a custom component.