efficient-kan icon indicating copy to clipboard operation
efficient-kan copied to clipboard

Published 20 hours ago • verified publisher icon Blealtan

time complexity

Open Aniosss opened this issue 9 months ago • 1 comments

Hey, I want to use your implementation, do you know how much slower the learning can be compared to nn.linear?

Aniosss avatar May 06 '24 12:05 Aniosss

Some benchmarks that might be helpful: https://github.com/GistNoesis/FusedFourierKAN/issues/4

akaashdash avatar May 07 '24 02:05 akaashdash

The activation is a bit (or more, if your VRAM bandwidth is limited) slower, and the number of parameters is (by default) 5x over nn.Linear with the same input and output size, so do the FLOPs count. I didn't do real benchmarks.

Also, AFAIK chebbykan is doing the same way as my impl, which is also the second best in the benchmark by @Jerry-Master mentioned above. The results seem reasonable to me.

Blealtan avatar May 17 '24 18:05 Blealtan