Activation function get_layer and model.forward() results are not same

[seyidcemkarakas]

I want to fully understand what activation plot tell us

Here is my code:

import numpy as np
import sympy as sp
import pandas as pd
import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score, mean_absolute_error

import torch
from kan import KAN

df2 = pd.read_csv("/Users/seyidcem/Desktop/KAN_GENEL/KAN_001/Admission_Predict_Ver1.1.csv")

# I used only 50 data points and 1 feature

df = df2.iloc[0:50] 
target_column_name = "Chance of Admit "
X = df[list(df.columns.drop([target_column_name]+["Serial No."]))[0:1]]
y = df[target_column_name]

# Split whole data to train and remainings
X_train, X_temp, y_train, y_temp = train_test_split(X, y, test_size=0.2, random_state=0)

# Split remainings data to val and test
X_val, X_test, y_val, y_test = train_test_split(X_temp, y_temp, test_size=0.5, random_state=0)

# Covert data to torch tensor
train_input = torch.tensor(X_train.to_numpy(), dtype=torch.float32)
train_label = torch.tensor(y_train.to_numpy()[:, None], dtype=torch.float32)
val_input = torch.tensor(X_val.to_numpy(), dtype=torch.float32)
val_label = torch.tensor(y_val.to_numpy()[:, None], dtype=torch.float32)
test_input = torch.tensor(X_test.to_numpy(), dtype=torch.float32)
test_label = torch.tensor(y_test.to_numpy()[:, None], dtype=torch.float32)

dataset = {
    'train_input': train_input,
    'train_label': train_label,
    'val_input': val_input,
    'val_label': val_label,
    'test_input': test_input,
    'test_label': test_label
}

# Create KAN
model = KAN(width=[len(X.columns),1], grid=5, k=2)

# Train KAN
results = model.train({'train_input': train_input, 'train_label': train_label, 'test_input': val_input, 'test_label': val_label},
                      opt="LBFGS", steps=50, loss_fn=torch.nn.MSELoss()) 

and here is:

model.plot()

l = 0
i = 0
j = 0

inputs = model.spline_preacts[l][:,j,i]
outputs = model.spline_postacts[l][:,j,i]
# they are not ordered yet
rank = np.argsort(inputs)
inputs = inputs[rank]
outputs = outputs[rank]
plt.plot(inputs, outputs, marker="o")

This model only contains 1 activation layer so this is my only activation layer.

After that I wanna try this with just 1 single data point:

model.forward(torch.tensor([[327]])).detach()
>>> tensor([[0.8054]])

but when I run this:

model.get_range(0,0,0)
>>> 
x range: [295.00 , 327.00 ]
y range: [0.60 , 0.83 ]

I see then X = 327 corresponds 0.83 but model.forward doesnt predicts like that. Why ? What this plot tells me?

Thanks