Aleksa Gordić

icon indicating a website link https://gordicaleksa.com/

icon company ex-DeepMind, ex-Microsoft icon location San Francisco icon location Flirting with LLMs. Tensor Core maximalist. If I say stupid stuff it's not me it's my prompt.

Results 43 comments of Aleksa Gordić

Pretraining (with CPUs)

Hey @bitmarkcc! Did you follow the README? You should first run the Python code, it'll generate all the necessary bin/state files before you run C/CUDA code. If something is not...

is max_seq_len configurable or hardcoded parameter?

yeah, we're focused on using the exact setup of gpt-2/3 right now, the idea is not to make this as configurable as possible, many things are hardcoded which goes against...

Modal script - benchmarking, profiling and libraries

I won't be able to help you with Modal, but I'll just say that our goal is ultimately to have custom CUDA kernels that outperform cuDNN. So you should, ideally,...