Kolmogorov-Arnold Networks (KANs), inspired by the Kolmogorov-Arnold representation theorem [4], are a novel class of neural networks characterized by learnable activation functions on edges instead of fixed activation functions on nodes. While KANs have demonstrated superior performance over traditional Multi-Layer Perceptrons (MLPs) in tasks requiring high-dimensional function approximation, their performance can be further optimized through effective initialization strategies and the introduction of residual connections. In this paper, an enhancement of KANs is proposed by combining Xavier initialization with residual activations. Xavier initialization ensures proper weight scaling, preventing vanishing or exploding gradients during training, while residual activations enable faster convergence and more efficient training of complex models. Through experimental evaluation on synthetic function approximation tasks, it demonstrates that these improvements yield faster convergence, better generalization, and increased robustness in training KANs.

Kolmogorov-Arnold Networks (KANs) Xavier Initialization Residual Activations Deep Learning