The Perceptron: Building Block of Neural Networks

The perceptron is the simplest form of a neural network - a single artificial neuron.

Biological Inspiration

Just as biological neurons receive signals through dendrites, process them, and fire output through axons, artificial neurons:

1. Receive inputs (x₁, x₂, ..., xₙ) 2. Apply weights (w₁, w₂, ..., wₙ) 3. Sum them up and add bias 4. Apply activation function 5. Produce output

Mathematical Formulation

y = f(Σ wᵢxᵢ + b)

Where: - xᵢ: Input features - wᵢ: Learnable weights - b: Bias term - f: Activation function - y: Output

Python Implementation

import numpy as np
class Perceptron:
    def __init__(self, n_inputs):
        
Initialize weights randomly
        self.weights = np.random.randn(n_inputs) * 0.01
        self.bias = 0
    
    def forward(self, x):
        
Weighted sum + bias
        z = np.dot(x, self.weights) + self.bias
        
Step activation (original perceptron)
        return 1 if z > 0 else 0
    
    def train(self, X, y, lr=0.1, epochs=100):
        for _ in range(epochs):
            for xi, yi in zip(X, y):
                prediction = self.forward(xi)
                
Update rule: w += lr * (y - pred) * x
                error = yi - prediction
                self.weights += lr * error * xi
                self.bias += lr * error
AND gate example
X = np.array([[0,0], [0,1], [1,0], [1,1]])
y = np.array([0, 0, 0, 1])
p = Perceptron(2)
p.train(X, y)

Limitations

The perceptron can only learn linearly separable patterns. It cannot learn XOR!