Logging in `on_test_epoch_end` with multiple dataloaders

[pschroeppel]

Bug description

Hello, I am running test() with multiple dataloaders and do the logging in on_test_epoch_end(). I basically follow https://github.com/Lightning-AI/pytorch-lightning/pull/16520 regarding the "Example with two DataLoaders".

In my case, I log different metrics for the different dataloaders (because each dataloader corresponds to a set set and I directly put the test set name into the logged name). For example:

• dataloader 1 logs clevr3d_cropped/ari
• dataloader 2 logs clevr3d/ari

However, the final table that is printed by PyTorch-Lightning, shows each logged value duplicated for each of the two dataloaders, which is incorrect:

        Test metric               DataLoader 0               DataLoader 1
─────────────────────────────
     clevr3d/ari         0.6213954091072083         0.6213954091072083
 clevr3d_cropped/ari      0.5558694005012512         0.5558694005012512
─────────────────────────────

Correctly, the table should look as follows:

        Test metric               DataLoader 0               DataLoader 1
─────────────────────────────
     clevr3d/ari         0.6213954091072083                -
 clevr3d_cropped/ari           -                          0.5558694005012512
─────────────────────────────

I think the problem is that self.log() does not know for which dataloader the log is and therefore logs for both dataloaders.

I also tried appending the dataloader index in the log, as described in the example mentioned above, but this didn't change anything.

My question now is: is this expected behaviour, or is it a bug, or am I using PyTorch-Lightning incorrectly?

Thanks and best, Philipp

What version are you seeing the problem on?

v2.5

Reproduced in studio

No response

How to reproduce the bug

Error messages and logs

# Error messages and logs here please

Environment

Current environment

#- PyTorch Lightning Version (e.g., 2.5.0):
#- PyTorch Version (e.g., 2.5):
#- Python version (e.g., 3.12):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):

More info

No response