reddyn12

> So this is unreviewable by reading. How have you tested that that's 0 functional changes? I tested against https://github.com/reddyn12/tinygrad/blob/upat_speed/test/test_pattern_matcher.py Unable to get it merged to master, but they pass.

Beam 3 BS 52 OOM Beam 3 BS 32 1.3s but 37, 28, 28, 28 gb used in gpu This is still proportional to BEAM 2 BS 52 step 2s

School is on a training run rn, no available compute. Can you do a run on a tiny tn for atleast 1 epoch? Expected scores: Epoch 1: 0.23 Epoch 2:...

@chenyuxyz can you run to get a step-time for me to compare to?

After the merge, getting NAN. Doing a run on a pre merge branch: https://wandb.ai/schlimeszn/RetinaNet/runs/0xl12h6e?nw=nwuserschlime. It's on epoch 3/4 and the first 2 epochs match Nvidia's submission. This is fp32, batch...

It errors: ``` python3 test/test_tensor_variable.py TestTensorVariable.test_symbolic_mean_2d_add F ====================================================================== FAIL: test_symbolic_mean_2d_add (__main__.TestTensorVariable.test_symbolic_mean_2d_add) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/nikhil/Code/Fun/tinygrad/test/test_tensor_variable.py", line 68, in test_symbolic_mean_2d_add t = Tensor.ones(2, 2).contiguous().reshape(vv2+add_term, vv+add_term) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File...

@geohot I cleaned up the tests, but now we are testing addition and meshgrid functionality in the same test, is that fine?

self.module is never instantiated in the init method

So it works with N=16, but that doesn't explain why the code runs with GPU=1. If the data was too big, shouldn't it fail in both GPU and CUDA =...

I put in the .realize between softmax and argmax for the time being and skipped the test in CI, should I add a benchmark run like with gpt?