Module 3 — Neural Networks & Deep Learning

Artificial neural networks are inspired by biological brains. A biological neuron receives signals through dendrites, processes them in the cell body, and fires an output through the axon when the combined signal exceeds a threshold.

The artificial equivalent — the perceptron — takes multiple inputs, multiplies each by a weight (representing its importance), sums them, applies an activation function, and produces an output. A single perceptron can learn simple patterns, but real power comes from connecting many perceptrons in layers.

A feedforward neural network consists of an input layer (receives data), one or more hidden layers (where learning happens), and an output layer (produces predictions). Each layer transforms the data into increasingly abstract representations. The first layer might detect simple features (edges in an image), middle layers combine these into more complex patterns (shapes, textures), and final layers recognise high-level concepts (faces, objects).

Backpropagation is the learning algorithm: after the network makes a prediction, the error is calculated and propagated backward through the network, adjusting weights to reduce the error. Over many iterations with many examples, the network learns to make accurate predictions. This is essentially the machine "learning" — adjusting millions of internal parameters to minimise prediction errors.

Convolutional Neural Networks (CNNs) are specialised for processing grid-like data, especially images. They use filters (small pattern detectors) that slide across the image, detecting features like edges, textures, and shapes. Deeper layers detect increasingly complex features — from edges to eyes to faces. CNNs power image recognition, medical imaging analysis, autonomous vehicle vision, and facial recognition.

Recurrent Neural Networks (RNNs) are designed for sequential data — text, speech, time series. They maintain a "memory" of previous inputs, allowing them to understand context. However, standard RNNs struggle with long sequences — they tend to forget earlier inputs (the "vanishing gradient" problem). LSTMs (Long Short-Term Memory) networks solved this with explicit memory mechanisms.

Transformers (Vaswani et al., 2017) revolutionised AI by introducing the attention mechanism — the ability to look at all parts of an input simultaneously and determine which parts are most relevant to each other. This parallel processing made them far more efficient than RNNs for long sequences.

Transformers power the current generation of large language models: GPT (Generative Pre-trained Transformer), BERT, Claude, and Gemini are all transformer-based. They're trained on massive text datasets and learn statistical patterns of language so well that they can generate coherent, contextually appropriate text — what appears to be "understanding" but is actually very sophisticated pattern matching.