dbl

I just started using it. I got this working... ``` def run_on_problem(problem, verbosity=0, multiprocessing=True): """ Takes in a problem and returns a tuple: (equations, best predicted equation, actual equation) """...

How did you get the loss to '0.4' on 'tinystories'? I'm still at 3.5 after ~2,000 iterations. ``` # data batch_size = 8 # if gradient_accumulation_steps > 1, this is...

I was able to train an llama2.c model with 12 heads and 12 layers with data from Covid-19 research papers (https://www.kaggle.com/datasets/allen-institute-for-ai/CORD-19-research-challenge/data) using device='mps' for 25,000 epochs without getting NaN's or...

I installed OpenCL-CLHPP E.g. https://github.com/KhronosGroup/OpenCL-CLHPP ``` % make test Running tests... Test project /Users/davidlaxer/OpenCL-CLHPP/build Start 1: test_openclhpp_120 1/45 Test #1: test_openclhpp_120 ............................................................... Passed 0.11 sec Start 2: test_openclhpp_120_CL_HPP_ENABLE_EXCEPTIONS 2/45 Test...

I 'hacked' /Users/davidlaxer/pytorch_dlprim/dlprimitives/include/dlprim/opencl_include.hpp: ``` # ifdef __APPLE__ //# include # include ``` Ran: ``` % cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DOCL_PATH="/Users/davidlaxer/OpenCL-CLHPP/include/" -DCMAKE_PREFIX_PATH=/Users/davidlaxer/anaconda3/envs/AI-Feynman/lib/python3.10/site-packages/torch/share/cmake/Torch .. $ make $ make install ``` It built and installed...

``` % python mnist.py --device ocl:0 Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ../data/MNIST/raw/train-images-idx3-ubyte.gz 100%|████████████████████████████| 9912422/9912422 [00:02

Correct! ``` % python mnist.py --device ocl:1 Using device: ocl:1 Accessing device #1:AMD Radeon Pro 5700 XT Compute Engine on Apple Train Epoch: 1 [0/60000 (0%)] Loss: 2.326377 Train Epoch:...

'MPS' is Apple's Metal Performance Shader framework. PyTorch now supports 'MPS' as a backend (e.g. - CUDA, MPS, ...) https://pytorch.org/docs/stable/notes/mps.html How do I run test.py? In test.py, when I set...

``` % ./clinfo Number of platforms 1 Platform Name Apple Platform Vendor Apple Platform Version OpenCL 1.2 (Dec 16 2022 20:35:20) Platform Profile FULL_PROFILE Platform Extensions cl_APPLE_SetMemObjectDestructor cl_APPLE_ContextLoggingFunctions cl_APPLE_clut cl_APPLE_query_kernel_names...

``` % python dlprimitives/tools/validate_network.py --device privateuseone:1 --benchmark --train --model resnet50 --batch 32 /Users/davidlaxer/anaconda3/envs/AI-Feynman/lib/python3.10/site-packages/torchvision-0.15.0a0+8985b59-py3.10-macosx-10.9-x86_64.egg/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use...