Module 1 — Understanding Large Language Models

Large Language Models (LLMs) are neural networks trained on massive text datasets to predict the next word in a sequence. Understanding how they work — even at a conceptual level — is essential for effective prompting.

Tokenisation: LLMs don't read words — they process tokens, which are chunks of text (roughly ¾ of a word on average). "Understanding" becomes two tokens: "Under" + "standing." This matters because LLMs have token limits (context windows), and pricing is typically per token.

The attention mechanism (the "T" in GPT stands for Transformer) allows the model to weigh the importance of every word in relation to every other word. When generating a response to "The cat sat on the ___", the model attends to "cat," "sat," and "on" to predict "mat" or "floor." In longer texts, attention allows the model to connect related concepts across paragraphs.

Autoregressive generation: LLMs generate text one token at a time, using everything generated so far as context for predicting the next token. Each token is selected from a probability distribution over the entire vocabulary. This is why the same prompt can produce different outputs — the model samples from probabilities rather than always choosing the most likely token.

The LLM landscape is evolving rapidly. Major models include:

GPT-4/GPT-4o (OpenAI): Strong reasoning, broad knowledge, multimodal (text + images). The benchmark for general-purpose AI.

Claude (Anthropic): Excellent at nuanced, long-form text. Known for careful, balanced responses and strong safety features.

Gemini (Google): Multimodal from the ground up (text, images, audio, video). Excels at tasks combining modalities.

LLaMA / Mistral (Meta / Mistral AI): Open-source models that can be fine-tuned and run locally, offering data privacy advantages.

Generation parameters control how the model produces text:

Temperature (0-2): Controls randomness. Low temperature (0-0.3) = deterministic, focused, factual. High temperature (0.7-1.2) = creative, diverse, unpredictable. Use low for analysis and facts; high for brainstorming and creative writing.

Top-p (nucleus sampling) (0-1): Limits token selection to those whose cumulative probability exceeds p. Top-p of 0.9 means the model considers only the top 90% most likely tokens, ignoring unlikely options.

Max tokens: Limits response length. Set it appropriately for your use case — too low truncates useful output; too high wastes resources.

Frequency/presence penalty: Reduces repetition by penalising tokens that have already appeared.