pykan
pykan copied to clipboard
Updated auto_symbolic to include a configurable threshold & simplicity weight
Dear Ziming, even though KAN 1.0 was an impressive work on its own, I believe that you managed to up the ante even further with KAN 2.0. Impeccable work, I will keep following the updates and working with KANs! This PR is a proposition:
-
Motivation: After trying to train relatively small KANs to fit relatively simple expressions, I noticed that
auto_symbolic
would sometimes produce seemingly nonsensical results. For example, when plotting the KAN, a certain edge would clearly correspond to a sine function, howeverauto_symbolic
would set it to zero. Upon further investigation, I realized thatauto_symbolic
callssuggest_symbolic
with the default value for theweight_simple
parameter, which prioritizes low complexity over high R^2 values. -
Proposition: I added the
weight_simple
as a parameter of theauto_symbolic
function as well, so that a user can choose how to prioritize symbolic fixing across the entire architecture. Usingsuggest_symbolic
andfix_symbolic
manually for each edge is tedious, especially in large architectures. Additionally, I introduced ar2_threshold
parameter which can control which edges to replace with symbolic expressions based on the achieved R^2. If the threshold is not reached, the user is prompted to either keep training until a better fit is found, or simply lower the threshold. Settingr2_threshold = 0.0
reproduces the original implementation, so there is complete "backward compatibility".
Kind regards, Spyros Rigas.