aim icon indicating copy to clipboard operation
aim copied to clipboard

Group statistics aren't being calculated

Open ADH-LukeBollam opened this issue 9 months ago • 2 comments

🐛 Bug

I record some final statistics into aim so I could compare results across experiments. These stats are measured outside training, so simply have a value at step 0 for each model. The charts look like this:

Image

Even though the charts have multiple values, I'm not getting any group stats

Image

To reproduce

This is how I'm logging these final stats:

    aim_logger.track(tp, name=f'{args.metric} True Positive', context={"subset": "test"})
    aim_logger.track(tn, name=f'{args.metric} True Negative', context={"subset": "test"})
    aim_logger.track(fp, name=f'{args.metric} False Positive', context={"subset": "test"})
    aim_logger.track(fn, name=f'{args.metric} False Negative', context={"subset": "test"})

    aim_logger.track(TotalErrors, name=f'{args.metric} TotalErrors', context={"subset": "test"})
    aim_logger.track(F1Score, name=f'{args.metric} F1Score', context={"subset": "test"})
    aim_logger.track(Precision, name=f'{args.metric} Precision', context={"subset": "test"})
    aim_logger.track(Recall, name=f'{args.metric} Recall', context={"subset": "test"})
    aim_logger.track(Specificity, name=f'{args.metric} Specificity', context={"subset": "test"})
    aim_logger.track(Accuracy, name=f'{args.metric} Accuracy', context={"subset": "test"})

Expected behavior

Environment

  • Aim Version 3.27.0
  • Python version 3.9
  • pip version
  • OS Ubuntu

Additional context

ADH-LukeBollam avatar Mar 19 '25 03:03 ADH-LukeBollam

Hey @ADH-LukeBollam! Thank a lot for opening the issue. There is already an issue opened on this: https://github.com/aimhubio/aim/issues/3293. Let's keep this one as well, as it's more descriptive of the underlying issue. Just wanted you to know that I'm already working on this, and will try to squeeze it in the upcoming 3.28.0 release, if not I'll make sure to ship a separate patch for this.

mihran113 avatar Mar 19 '25 11:03 mihran113

Hi Mihran, I'm trying out the latest version with group statistics, I'm not sure that the min and max are correct, unless I'm misunderstanding what they mean.

Image

ADH-LukeBollam avatar May 14 '25 07:05 ADH-LukeBollam