Cg Lai

Hello, Thanks for working this problem. It helps me a lot. I can install @mvitez 's code to torch7, but it has a problem when I use torch7. torch/install/bin/luajit: torch/install/share/lua/5.1/trepl/init.lua:384:...

Thanks @williamBlazing for the detailed explanation. Glad to hear you have a plan to do that.

Thanks @davidwendt . It was working to convert it to decimal and write it as csv.

Yes, thanks again!

I got the same problem when enable modulate kernel.

Thanks @feifeibear for the quick reply. I'll check the benchmark results and let me know if you have any updates. Many thanks!

Hello @feifeibear, thanks for the reply and new code. I'll check it, For CPU memory in one node, I usually set 1.9T for the 30B model on 8 GPUs.

Thanks @JThh for the reply. Currently we do not have GPU memory savings data, but we can do more testings for that. Do you have any ideas regarding the CPU...

@ver217 I trained opt 66b on the single A100.

Thanks @jdye64 . I have updated the CUDA version to 11.6. It is good to compile in C++, but I got the error when I import the library in the...