lecture-jax icon indicating copy to clipboard operation
lecture-jax copied to clipboard

Replace `%time` with `%timeit`

Open HumphreyYang opened this issue 8 months ago • 3 comments

@shizejin reported that there is a misleading comparison in the introduction to JAX:

Image

It might relates to the timing magic we used. %time is sometimes influenced other factors such as garbage collections, workload etc: https://stackoverflow.com/questions/17579357/time-time-vs-timeit-timeit

I think %timeit might be a more robust way to compare the results, which is also used in JAX documents: https://docs.jax.dev/en/latest/quickstart.html

The result under %timeit can be found here: Image

HumphreyYang avatar Apr 02 '25 13:04 HumphreyYang

Thanks @HumphreyYang and thanks @shizejin for reporting!

(@HumphreyYang always puts in great PRs, so the whole issue is clear at a glance. This is super valuable to me.)

I was trying to avoid timeit because people who don't understand ipython get the impression that the code is slow.

But it seems this is causing problems.

@HumphreyYang , let's do as you recommend. Would you mind to put the fix in?

jstac avatar Apr 07 '25 05:04 jstac

Many thanks @jstac for the suggestion! I will submit a FIX for this.

I was trying to avoid timeit because people who don't understand ipython get the impression that the code is slow.

I think this is a very valid concern as well. The reader sees that the number is small but notices that the code takes longer to run.

I will add a brief note below the first use of %timeit, explaining that it runs the line multiple times to provide a robust comparison.

Given this concern, should we change %timeit globally or only in the introductory lecture?

I think the issue is less noticeable when the runtime differences are large.

HumphreyYang avatar Apr 07 '25 06:04 HumphreyYang

I agree @HumphreyYang . Let's change it only in the introductory lecture and with the note that you mention. Thanks.

jstac avatar Apr 07 '25 07:04 jstac

Fixed by using JAX==0.6.0

HumphreyYang avatar Apr 30 '25 01:04 HumphreyYang