Jacky Lee

Thanks @NielsRogge! This worked for me: ``` import torch import requests from PIL import Image, ImageDraw from transformers import AutoProcessor, AutoModelForZeroShotObjectDetection checkpoint="google/owlv2-base-patch16-ensemble" model = AutoModelForZeroShotObjectDetection.from_pretrained(checkpoint) processor = AutoProcessor.from_pretrained(checkpoint) url =...

The [GitHub-hosted runners don't have a GPU](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources), so CL uses the CPU for GPU tests. I ran a [modified test](https://github.com/jla524/tinygrad/actions/runs/4090705359/jobs/7054302509) to confirm this. ``` >Run python -c "import pyopencl as...

Sounds good. Here are the slowest tests on GPU: ``` 97.35s call test/test_train.py::TestTrain::test_efficientnet 44.74s call test/test_ops.py::TestOps::test_conv2d 40.05s call test/test_train.py::TestTrain::test_resnet 32.05s call test/test_mnist.py::TestMNIST::test_conv 30.07s call test/test_onnx.py::TestOnnxModel::test_benchmark_openpilot_model 27.15s call test/test_mnist.py::TestMNIST::test_sgd 22.70s call...

Most of the time is spent on `optim.step()`, and it looks like [Adam is much slower than SGD](https://discuss.pytorch.org/t/optimizer-step-the-slowest/90820). With Adam: ``` test/test_train.py::TestTrain::test_efficientnet stepping with 5.3M params bs 2 sampling took...

I've added some tests to check if the distributions match torch. We can add something like this in `test/test_randomness.py`. ``` def test_kaiming_uniform(self): self.assertFalse(normal_test(Tensor.glorot_uniform)) self.assertTrue(equal_distribution(Tensor.kaiming_uniform, lambda x: torch.nn.init.kaiming_uniform_(torch.empty(x)), lambda x: (np.random.rand(*x)...

I ran the following tests and they failed due to shape mismatch. Should the shapes match? ``` def test_std_axis(self): # AssertionError: shape mismatch (tinygrad)(1,) != (torch)(65, 85) helper_test_op([(45,65,85)], lambda x:...

> I found the problem!! According to [1], Pytorch uses Bessel's correction to measure std which is different from Numpy std. To implement like Numpy, setting `unbiased=False`, fixes everything. >...

These tests passed but I had to increase the tolerance on some of them. Particularly when `unbiased=False` and (`dim=0` or `dim=2`). ``` def test_std_axis(self): helper_test_op([(45, 65, 85)], lambda x: torch.std(x,...

> How is the score looking? I got a mean DICE score of 0.87 just now Edit: 0.86632 to be exact, it barely meets the target of 0.86330

> Hmm, so there's still an `nn.InstanceNorm` but tests for it were removed? my bad, I put the tests back