tutorials icon indicating copy to clipboard operation
tutorials copied to clipboard

Fix pre/post-training evaluation to use same batch in nn_tutorial

Open patrocinio opened this issue 1 month ago • 1 comments

The tutorial was comparing loss on different batches:

  • Pre-training: evaluated on first 64 instances (batch 0)
  • Post-training: evaluated on last batch from training loop

This made the comparison misleading as it wasn't measuring improvement on the same data.

Changes:

  • Save the initial batch (xb_initial, yb_initial) after first evaluation
  • Use the saved initial batch for post-training evaluation
  • Added clarifying comment about fair comparison
  • Now both evaluations use the same data (first 64 training instances)

This provides an accurate before/after comparison showing the model's improvement on the same batch of data.

Fixes #3666

Description

Checklist

  • [ ] The issue that is being fixed is referred in the description (see above "Fixes #ISSUE_NUMBER")
  • [ ] Only one issue is addressed in this pull request
  • [ ] Labels from the issue that this PR is fixing are added to this pull request
  • [ ] No unnecessary issues are included into this pull request.

patrocinio avatar Nov 27 '25 00:11 patrocinio