In artificial neural networks, an activation function is a mathematical operation applied to the weighted sum of inputs at a neuron, transforming it into an output that introduces non-linearity, thereby enabling the network to model complex, non-linear relationships in data.[1] These functions are essential components of neural architectures, as without them, multi-layer networks would reduce to simple linear models incapable of capturing intricate patterns.[2]The concept of activation functions traces its origins to early models of biological neurons, notably the 1943 McCulloch-Pitts neuron, which employed a binary step function as its activation to simulate logical operations like AND and OR gates.[3] This threshold-based approach laid the foundation for computational neuroscience and inspired the 1958 perceptron by Frank Rosenblatt, which also utilized a