aim icon indicating copy to clipboard operation
aim copied to clipboard

Aggregated metrics not computed correctly

Open kirschnj opened this issue 1 year ago • 2 comments

🐛 Bug

Aggregated values (mean, median) are not displayed correctly.

To reproduce

Log several values for different contexts, where some values appear multiple times as below:

import aim
run = aim.Run(experiment="test")

for step in range(3):
    run.track(0, name="test", step=step, context=dict(context=0))
    run.track(0, name="test", step=step, context=dict(context=1))
    run.track(1, name="test", step=step, context=dict(context=2))

run.close()

In the UI, both median and mean are incorrectly shown as 0.5.

image

Expected behavior

In the example above, the mean per step is 0.33 and the median is 0.

Environment

  • Aim v3.25.0 on Linux

Additional context

The issue appears to be that only unique values in the group are used to compute the mean/median. The issue doesn't appear with distinct values (e.g. 0, 0.2, 1).

kirschnj avatar Nov 01 '24 14:11 kirschnj

Hey @kirschnj! Thanks a lot for noticing this. 🙌 I'll open the PR with a fix shortly.

mihran113 avatar Nov 07 '24 16:11 mihran113

Hey @kirschnj! The fix for this has been shipped with the latest release of aim: version 3.27.0. Let me know if everything works properly.

mihran113 avatar Dec 25 '24 13:12 mihran113