Andrej comments

Results 373 comments of


                                            Andrej

Gain another 10-20%+ on CPU performance on gcc by moving -fno-finite-math-only to only gelu_backwards

So maybe this is ok to merge... 1 it looks a little funny is there no way to combine the double nested if into one condition? 2 i think a...

Gain another 10-20%+ on CPU performance on gcc by moving -fno-finite-math-only to only gelu_backwards

I'm sorry this is too weird and ugly to merge I think. Can someone try alternative strategies? For example tanh can be written as a function of exp quite trivially,...

CPU: atomicAdd(float*)

Btw some of these kernels i was being very lazy, e.g. encoder_backward, it's very possible we don't need to use atomics at all. Personally I'm a bit sus about them,...

CPU: atomicAdd(float*)

(The repo is very new, I was just trying to get it to check w.r.t. PyTorch quickly, and then some of these kernels don't make it the biggest time sink...

Simple float CUDA port

Thank you for the PR! I'm traveling right now so a bit slower on reply, but looking forward to taking a look

meaning of having if-statments at tx_fetcher's fetch method

I had the same question, it is unfortunate that this is not commented. `raw[4] == 0` denotes that this is a Segwit transaction, where bytes 5, 6 are the Segwit...

Give tqdm some love :)

hahahaha :D HMMM

Give tqdm some love :)

ok fine. widely requested change :D

Support GQA export, better run.c, Support tinyllama-1.1B

This is cool, I wasn't aware of the TinyLlama 1.1B run. Sounds very nice and useful for this repo to support. Are there any notable architectural changes in it? This...

MultiGPU training hangs

Bleh I can reproduce this 🤦‍♂️ . Would most likely be helpful to harden our CI to include multi-GPU training, as it falls outside of it. iirc @Ricardicus helped set...