The results are different from hellokan.ipynb

[yt7589]

Hi. I had followed the instruction exactly. When I run the hellokan.ipynb it worked. But the result is different from the one in the original version. For example the prouned network had more notes than the original one: The original one: My version was:

For the symbolic formula, the original one is: My version was:

What's wrong? Please help me to fix this problem.

[KindXiaoming]

Hi, it's likely that my random seed is not set properly or the notebook tutorials are a bit lagged behind the code. Does random seed work for you (with the same random seed, do you always get the same results)?

Also, it looks like your network is promising too, maybe just need to set the threshold in model.prune() to be larger model.prune(threshold=5e-2) might do the job. by default threshold = 1e-2. By increasing the threshold, the second neuron in the hidden layer will hopefully be pruned away. Or you simply need train longer or pump up lamb a little bit.

[Kodp]

I have same problem. Can the author give random seeds?

[Kodp]

Hi, it's likely that my random seed is not set properly or the notebook tutorials are a bit lagged behind the code. Does random seed work for you (with the same random seed, do you always get the same results)?

Also, it looks like your network is promising too, maybe just need to set the threshold in model.prune() to be larger model.prune(threshold=5e-2) might do the job. by default threshold = 1e-2. By increasing the threshold, the second neuron in the hidden layer will hopefully be pruned away. Or you simply need train longer or pump up lamb a little bit.

Setting the threshold to 5e-2 works.

[Fakasht10]

I had a similar issue, in the hello Kan example the loss was not going to below 1e-4...I made two mistakes i did not install the suggested packages in a new environment and the second one i dont have cuda on my pc so it was running on cpu and in single precision I think. So I installed in the new environment the exact required packages and i just put this line in the beginning of the notebook

import torch

Set the default tensor type to double precision

torch.set_default_dtype(torch.float64)

and then the loss was down to 1e-10 or less.

On a side note, i didnt see any examples with multi input miulti output...I dont think that kan is limited to that regauged to just one output variable? I haven't seen any symbolic regression packages with multi-input multi output only multi input single output. Also, I wonder if the KAN network is naturally well suited to discover equation like SINDy...or if it can be used as a right hand side of a NODE neural ode or in a UDE setting....which brings another thing I am wondering about are there any advantage to coding up KAN in Julia to take advantage of the Lux/Flux ode solvers for NODE type problems...

[yyugogogo]

I had a similar issue, in the hello Kan example the loss was not going to below 1e-4...I made two mistakes i did not install the suggested packages in a new environment and the second one i dont have cuda on my pc so it was running on cpu and in single precision I think. So I installed in the new environment the exact required packages and i just put this line in the beginning of the notebook

import torch

Set the default tensor type to double precision

torch.set_default_dtype(torch.float64)

and then the loss was down to 1e-10 or less.

On a side note, i didnt see any examples with multi input miulti output...I dont think that kan is limited to that regauged to just one output variable? I haven't seen any symbolic regression packages with multi-input multi output only multi input single output. Also, I wonder if the KAN network is naturally well suited to discover equation like SINDy...or if it can be used as a right hand side of a NODE neural ode or in a UDE setting....which brings another thing I am wondering about are there any advantage to coding up KAN in Julia to take advantage of the Lux/Flux ode solvers for NODE type problems...

Hi there! Do you know why KAN throws this error after adding torch.set_default_dtype(torch.float64)? Sorry I'm new to torch.

[inkydragon]

@yyugogogo This patch works for me.

patch

-            self.scale_base = torch.nn.Parameter(torch.FloatTensor(scale_base).to(device)).requires_grad_(sb_trainable)
+            self.scale_base = torch.nn.Parameter(torch.tensor(scale_base.numpy()).to(device)).requires_grad_(sb_trainable)

https://github.com/KindXiaoming/pykan/blob/27b4640aeee010dabb34264e6c7d7272bcf130f8/kan/KANLayer.py#L126

ref： https://discuss.pytorch.org/t/typeerror-expected-tensoroptions-dtype-float-device-cpu-layout-strided-requires-grad-false-default-pinned-memory-false-default-memory-format-nullopt/159558

[yyugogogo]

@inkydragon Thanks! That works!