How to use KAN to fit a implicit function?

[huangzhouzhou250]

Hi, Thank you very much for your work. I want to use KAN to fit an implicit function, like the spherical surface x^2+y^2+z^2=1. Could you give me any advice?

[AlessandroFlati]

Well, I would treat as a higher-dimensional problem and/or as a classification problem. (You may want to check out https://kindxiaoming.github.io/pykan/Examples/Example_3_classfication.html)

In the example at hand, I would generate a dataset based on the function

def f(x): 
    return x[:, [0]] ** 2 + x[:, [1]] ** 2 + x[:, [2]] ** 2

then train the KAN as usual, which should give you a machine-precision model that can approximate the function itself (very easily). Then you can simply train a classifier or even a simple algorithm that checks if the output of the model is 1 (well, I would approach with a abs(output - 1) < eps) or not.

Otherwise you can just treat it as a classification problem all encompassed, as in the link provided.

[jloveric]

I've worked on a very similar algorithm to KAN using piecewise lagrange polynomials, and implicit representation is the most promising application I've encountered so far. Unfortunately, NERFs have basically been replaced by guassian splatting so it would probably be some other implicit representation that should be tested https://github.com/jloveric/high-order-implicit-representation showing simple image fitting using piecewise polynomial layers

[huangzhouzhou250]

Hi AlessandroFlati, Thanks for your advice. I tried in the past two days. The results are shown as follows.

1. Generated a dataset based on the function f(x): f(x)=x^2+y^2+z^2. Then trained the KAN as usual. As you said, it learned the f(x). Thanks for your work.
2. All values of the function need to be set to 0 since the variables satisfy f(x)=0. In the example, I generated the dataset based on sphere coordinates：

    v = np.random.rand(n_samples)
    theta = 2 * np.pi * u
    phi = np.arccos(2 * v - 1)
    x = np.sin(phi) * np.cos(theta)
    y = np.sin(phi) * np.sin(theta)
    z = np.cos(phi)
I trained KAN. The result is 
         25.61(0.17-` x2)^3+3454.24sin(2.44*x3)-6.76(3.71*x1-0.79)-30.89
The parameters of KAN are as follows 
       model = KAN(width=[3,1], grid=3, k=3)
3. I tried to use a  variably as the function of others. In the example, I set the z = g(x,y). I trained KAN.  The result is
           0.94-0.2 sin(-7.74(-x1-0.32)^3+1.7sin(1.49*x2-1.58)+4.8)

Could you give me any advice to optimize the results?

[AlessandroFlati]

I'm sorry, but I don't understand what you're trying to do with the last part (from model = KAN(width=[3,1]...). If the intent is to classify points belonging to x^2 + y^2 + z^2 = 1, then you're doing it wrong.

[huangzhouzhou250]

Hi AlessandroFlati, Thank you for your reminder。 The goal is to learn the expression of an implicit function f(x) and then generate new data. The process is as follows： 1 Collect the data to build the dataset. In my research, I collect data from demonstrations of a robot task. The variables of robot joints satisfy f(x)=0 in the task. Hence, these data are on a manifold. 2 Learn the expression of the f(x). I tried manifold learning methods, but no use.

The intent is to learn the expression f(x) when there are data satisfying f(x) =0. Could you give me any advice to use KAN to do it?

[KindXiaoming]

Hi I think this is related to unsupervised learning. An example: https://kindxiaoming.github.io/pykan/Examples/Example_12_unsupervised_learning.html

[huangzhouzhou250]

Hi, Thanks for your advice. I tried it today. I feel this network is so powerful. In the spherical surface example, the results are shown as follows, 1 The parameters of KAN are as

     model = KAN(width=[3,1,1], grid=3, k=3, seed=seed)
     train_num=500
     lib = ['x','x^2','x^3','x^4','sqrt']

 2 The expression of the final function is 

0.08*(-0.29*(-0.93*x_1 - 1)**2 - 0.23*(-1.0*x_2 - 1)**2 - 0.25*(-0.87*x_3 - 1)**2 - 1)**2 - 0.26

3 When i set

model.fix_symbolic(1,0,0,'sqrt',fit_params_bool=False) 

The result is

0.66*sqrt(0.38*(-0.93*x_1 - 1)**2 + 0.3*(-1.0*x_2 - 1)**2 + 0.33*(-0.87*x_3 - 1)**2 - 1)

It is very close to the function I want. Thanks for your work. Could you give me any advice about how to adjust the parameters of KAN network if I want a more precise result？

[wangmiaowei]

while I also test that, the spherical shape is good. but for more complex geometry shapes.. bad happens: could you give me some advice for that?