【Kolmogorov-Arnold网络替代多层感知机MLPs】KAN: Kolmogorov-Arnold Networks

KAN: Kolmogorov-Arnold Networks

论文地址
代码地址
知乎上的讨论（看一下评论区更正）

请添加图片描述

Abstract

Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes (“neurons”), KANs have learnable activation functions on edges (“weights”). KANs have no linear weights at all – every weight parameter is replaced by a univariate function parametrized as a spline. We show that this seemingly simple change makes KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving. Theoretically and empirically, KANs possess faster neural scaling laws than MLPs. For interpretability, KANs can be intuitively visualized and can easily interact with human users. Through two examples in mathematics and physics, KANs are shown to be useful collaborators helping scientists (re)discover mathematical and physical laws. In summary, KANs are promising alternatives for MLPs, opening opportunities for further improving today’s deep learning models which rely heavily on MLPs.

受Kolmogorov-Arnold 表示定理的启发，我们提出了Kolmogorov-Arnold网络（KANs）作为多层感知器（MLPs）的替代品。

MLP 在节点（“神经元”）上有固定的激活函数，而 KAN 在边缘（“权重”）上有可学习的激活函数。

KAN 完全没有线性权重–每个权重参数都由参数化为样条曲线的单变量函数代替。

我们的研究表明，这一看似简单的改变使得 KAN 在准确性和可解释性方面都优于 MLP。

就准确性而言，在数据拟合和 PDE 求解方面，更小的 KAN 可以达到与更大的 MLP 相当或更高的准确性。从理论和经验上讲，KANs 比 MLPs 拥有更快的神经缩放规律。
在可解释性方面，KANs 可以直观地可视化，并很容易与人类用户进行交互。

通过数学和物理学中的两个例子，KANs 被证明是帮助科学家（重新）发现数学和物理定律的有用合作者。总之，KANs 是 MLPs 有前途的替代品，为进一步改进当今严重依赖 MLPs 的深度学习模型提供了机会。

Accuracy

KANs have faster scaling than MLPs. KANs have better accuracy than MLPs with fewer parameters.
KAN 的扩展速度比 MLP 快。KAN 在参数较少的情况下比 MLP 更准确。

Example 1: fitting symbolic formulas

请添加图片描述

Example 2: fitting special functions

请添加图片描述

Example 3: PDE solving

请添加图片描述

Example 4: avoid catastrophic forgetting

请添加图片描述

Interpretability

KANs can be intuitively visualized. KANs offer interpretability and interactivity that MLPs cannot provide. We can use KANs to potentially discover new scientific laws.
KAN 可以直观地可视化。KANs 具有 MLP 无法提供的可解释性和互动性。我们可以利用 KAN 发现新的科学规律。