Iman Hosseini

[GLM-130B: An Open Bilingual Pre-trained Model](https://arxiv.org/abs/2210.02414) New results on applying quantization to GPT models. Do you have any suggestions -or say, a blueprint- on how to adapt the current GPT-J...

Internet connection was ok (I'd just cloned the repo), but since then I unplugged it _unsafely_ and now it doesn't show up in 'mdt devices' anymore. The light goes green,...

I reflashed and fixed.

Ok I hate git. How can I squash the commits? I'm doing as this says: https://stackoverflow.com/questions/5189560/how-do-i-squash-my-last-n-commits-together

I use conda command prompt, and I get stuck on "[Msg] Waiting for master." weirdly, it randomly works sometimes [everything works] but most of the time it just gets stuck...

win64. conda info: ``` active environment : r2 active env location : C:\Users\salva\miniconda3\envs\r2 shell level : 2 user config file : C:\Users\salva\.condarc populated config files : C:\Users\salva\.condarc conda version :...

A question regarding INT8 support: Q1. Almost all templated classes (e.g. https://github.com/NVIDIA/FasterTransformer/blob/6fddeac5f59ce4df380002aa945da57a0c8e878c/src/fastertransformer/models/gpt/GptDecoderLayerWeight.cc#L201) only support float or half: ``` template struct GptDecoderLayerWeight; template struct GptDecoderLayerWeight; ``` Assuming one wants to implement...

Not sure about how they did it, but this change: https://github.com/Guangxuan-Xiao/torch-int/pull/1/commits/2163a169748edff67586c2bf0158f4c7f0718fc6 includes an implementation for Gelu unit.

Thank you. Do you know of any open source tool which does that or any pointers on where to start making such a tool?

For M = 2000000, (on RTX 3060 Ti): ``` # for different n1/n2/n3 and m \in {1,2,4} ./cuperftest --N1=n1 --N2=n2 --N3=n3 --method=m ```  m=4 is much slower. 'spread_3d_block_gather' achieves...