What Are LLMs?

LLMs are machine learning models trained on vast amounts of text data. They use transformer architectures, a neural network design introduced in the paper “Attention Is All You Need”. Transformers excel at capturing context and relationships within data, making them ideal for natural language tasks.

1. Architectural Types of Language Models
(Expanded with Technical Nuances)

A. Decoder-Only (Autoregressive) Models
– Core Mechanism: Processes text sequentially from left-to-right using masked self-attention. Each token prediction depends only on previous tokens.
– Key Innovations:
– Sparse Attention (e.g., GPT-3’s block-sparse patterns) for long-context efficiency.
– Rotary Positional Embeddings (RoPE) in LLaMA for better positional encoding.
– Limitations: Struggles with bidirectional context understanding (e.g., fill-in-the-blank tasks).
– Examples: GPT-4 (345B params), Mistral 7B (sliding window attention), PaLM 2 (Google’s pathway scaling).

B. Encoder-Only (Autoencoding) Models
– Core Mechanism: Uses full bidirectional attention to reconstruct masked tokens (e.g., BERT’s 15 – Training Tricks:
– Dynamic Masking (RoBERTa): Changes masked tokens per epoch.
– Whole-Word Masking: Masks entire words for Chinese/Japanese.
– Use Cases:
– Sentence embeddings (e.g., SBERT for semantic search).
– Low-latency classification (DistilBERT’s 66

C. Encoder-Decoder (Sequence-to-Sequence) Models
– Hybrid Approach: Encoder processes input bidirectionally; decoder generates output autoregressively.
– Specialized Variants:
– T5: Treats all tasks as text-to-text (e.g., “translate English to German: …”).
– BART: Optimized for denoising (e.g., document reconstruction).
– Efficiency Trade-offs: 30-40

2. Training Objectives & Pretraining Strategies
(Beyond Basic Causal/Masked LM)

A. Multitask Pretraining
– FLAN-T5: Trained on 1,800+ instruction templates across 140 tasks.
– UniLM: Combines causal, masked, and seq2seq objectives in one model.

B. Reinforcement Learning from Human Feedback (RLHF)
– Process: Supervised fine-tuning → Reward modeling → PPO optimization.
– Critical for Alignment: Reduces harmful outputs by ~60

C. Denoising Objectives
– BART’s Approach: Randomly corrupts text (deletion, permutation, masking) and learns to reconstruct.
– PEGASUS: Specifically designed for summarization via gap-sentence generation.

3. Specialized LLMs & Emerging Categories

A. Vision-Language Models (VLMs)
– Architecture:
– Single-Stream (Flamingo): Interleaves image and text tokens in one transformer.
– Dual-Encoder (CLIP): Separate image/text encoders with contrastive learning.
– Breakthrough Models:
– GPT-4V: Processes images via vision encoder + LLM fusion.
– Kosmos-2: Grounds text to image regions (e.g., “click on the red car”).

B. Code-Specialized LLMs
– Training Data:
– StarCoder: 80+ programming languages from GitHub (1TB code).
– Code LLaMA: Infill-compatible (e.g., predicts missing code segments).
– Unique Features:
– Repository-Level Context: AlphaCode processes entire GitHub repos.
– Unit Test Execution: CodeT5+ validates outputs against test cases.

C. Domain-Specific LLMs
– Medical:
– Med-PaLM 2: Achieves 85 – BioBERT: Pretrained on PubMed abstracts.
– Legal:
– LegalGPT: Fine-tuned on 2M court opinions.
– Harvey AI: Used by Allen & Overy for contract review.

4. Multimodal & Embodied AI Frontiers

A. Audio-Language Models
– Whisper: ASR + translation via encoder-decoder.
– AudioPaLM: Merges speech and text tokenizers for voice assistants.

B. Robotics Integration
– RT-2: Uses VLMs to convert camera inputs to robot actions (“pick up the banana”).
– PaLM-E: Embodied model handling sensor data + language.

C. Agentic LLMs
– AutoGPT: Recursively decomposes goals into sub-tasks.
– Voyager: Minecraft AI that learns from environment feedback.

Comparative Summary Table

Core Components of an LLM

1. Tokenizer: Splits text into smaller units like words or subwords.
2. Embedding Layer: Converts tokens into dense vector representations.
3. Transformer Blocks: Layers that use self-attention mechanisms to process and understand input sequences.
4. Output Layer: Generates predictions, such as the next word in a sentence.

Now Let’s create the LLM step by step:

I shall use colab and we shall look at all the steps involved for this example we shall use “distilgpt2” model.

Model Name Definition:

This sets the variable model_name to the string “distilgpt2”

“distilgpt2” refers to a distilled (smaller, faster) version of the GPT-2 model created by Hugging Face

This is a pretrained language model that can generate human-like text

Loading the Tokenizer:

AutoTokenizer is a class from the Hugging Face transformers library that automatically selects the appropriate tokenizer based on the model name

from_pretrained(model_name) loads the tokenizer that was specifically trained with the “distilgpt2” model

The tokenizer handles:

Splitting text into tokens (words/subwords)

Converting tokens to numerical IDs (tokenization)

Converting numerical IDs back to text (detokenization)

Handling special tokens like [CLS], [SEP], etc.

Loading the Model:

• • AutoModelForCausalLM is a class that automatically selects the appropriate model architecture for causal language modeling

  • from_pretrained(model_name) downloads and loads:

    • The model architecture (in this case, a distilled GPT-2 architecture)

    • The pretrained weights (the knowledge the model learned during training)

  • This is a “causal” language model, meaning it’s designed to predict the next word in a sequence (used for text generation)

Key points about what happens under the hood:

• Both operations will download the model/tokenizer from Hugging Face’s model hub if they’re not already cached locally

• The downloaded files are stored in a cache directory (typically ~/.cache/huggingface)

• The model will be loaded in evaluation mode by default (no training/gradient computation)

• The tokenizer includes all the special tokens and vocabulary needed to preprocess text for this specific model

After these steps, you’ll have:

• A tokenizer ready to convert between text and token IDs

• A model ready to make predictions (generate text) based on input token IDs

Let me break down each step mathematically with clear explanations and key points regarding the output above.

Step 1: Original Text Processing
Text: "The weather is"

– This is a sequence of 3 words (including the space before “weather” and “is”).
– The tokenizer will split this into subword tokens based on the vocabulary.

Mathematical Representation:
– Let the input text be a string \( S = \) "The weather is".
– The tokenizer \( \mathcal{T} \) maps \( S \) to a sequence of tokens \( T \):
\[
\mathcal{T}(S) = T = [t_1, t_2, t_3] = \text{[‘The’, ‘Ġweather’, ‘Ġis’]}
\]
– Ġ indicates a preceding space in GPT-style tokenization.

Step 2: Tokenization
Output:
– Tokens: ['The', 'Ġweather', 'Ġis']
– Token IDs: tensor([[464, 6193, 318]])
– Attention Mask: tensor([[1, 1, 1]])

Mathematical Explanation:
1. Token → ID Mapping:
– The tokenizer has a vocabulary \( V \) where each token \( t_i \) is assigned a unique integer \( x_i \).
– The mapping is:
\[
\begin{cases}
t_1 = \text{‘The’} & \rightarrow x_1 = 464 \\
t_2 = \text{‘Ġweather’} & \rightarrow x_2 = 6193 \\
t_3 = \text{‘Ġis’} & \rightarrow x_3 = 318 \\
\end{cases}
\]
– The input sequence becomes:
\[
X = [x_1, x_2, x_3] = [464, 6193, 318]
\]
2. Attention Mask:
– Since all tokens are valid (not padding), the mask is [1, 1, 1].
– If padding were present, some positions would be 0.

Key Points:
– The tokenizer converts text into numerical IDs that the model can process.
– The attention mask helps the model ignore padding tokens during computation.

Step 3: Embedding Matrix
Output:
– Shape: torch.Size([50257, 768])
– Description: A matrix of size (vocab_size, embedding_dim).

Mathematical Explanation:
1. Embedding Matrix Definition:
– Let \( W_e \in \mathbb{R}^{V \times d} \), where:
– \( V = 50257 \) (vocabulary size)
– \( d = 768 \) (embedding dimension)
– Each row \( W_e[i] \) is the embedding vector for token ID \( i \).

2. Embedding Lookup:
– For each token ID \( x_i \), the embedding is:
\[
e_i = W_e[x_i]
\]
– For our input \( X = [464, 6193, 318] \), we get:
\[
\begin{cases}
e_1 = W_e[464] \\
e_2 = W_e[6193] \\
e_3 = W_e[318] \\
\end{cases}
\]
– The final embedded sequence is:
\[
E = [e_1, e_2, e_3] \in \mathbb{R}^{3 \times 768}
\]

Key Points:
– The embedding matrix is a trainable lookup table that maps discrete token IDs to continuous vectors.
– Each token is represented as a dense vector in \( \mathbb{R}^{768} \).
– Similar words tend to have similar embeddings (closer in vector space).

Step 4: Embedded Input Vectors
Output:
– Shape: torch.Size([1, 3, 768]) (batch_size=1, sequence_length=3, embedding_dim=768)
– First 5 elements shown for each token.

Mathematical Explanation:
1. Embedding Output:
– The embedded sequence is:
\[
E = \begin{bmatrix}
e_{1,1} & e_{1,2} & \cdots & e_{1,768} \\
e_{2,1} & e_{2,2} & \cdots & e_{2,768} \\
e_{3,1} & e_{3,2} & \cdots & e_{3,768} \\
\end{bmatrix}
\]
– The example shows:
\[
e_1 = [-0.0626, -0.0449, 0.0559, -0.0547, -0.1171, \dots]
\]
\[
e_2 = [0.1632, 0.1023, 0.0634, 0.1102, -0.0860, \dots]
\]
\[
e_3 = [-0.0006, 0.0075, 0.0307, -0.1343, -0.1336, \dots]
\]

2. Batch Dimension:
– Since the input is a single sequence, the output has shape (1, 3, 768).
– For a batch of size \( B \), the shape would be (B, seq_len, 768).

Key Points:
– The embeddings capture semantic and syntactic features of the tokens.
– These vectors will be fed into the transformer layers for further processing.
– The embedding step is differentiable, allowing gradients to flow back during training.

Summary of Mathematical Flow
1. Text → Tokens:
\[
S \rightarrow \mathcal{T}(S) = [t_1, t_2, t_3]
\]
2. Tokens → IDs:
\[
[t_1, t_2, t_3] \rightarrow [x_1, x_2, x_3]
\]
3. IDs → Embeddings:
\[
[x_1, x_2, x_3] \rightarrow [W_e[x_1], W_e[x_2], W_e[x_3]] = [e_1, e_2, e_3]
\]
4. Final Embedding Tensor:
\[
E \in \mathbb{R}^{1 \times 3 \times 768}
\]

Why This Matters
– Discrete → Continuous: Converts words into numerical vectors.
– Semantic Similarity: Words with similar meanings have closer embeddings.
– Downstream Processing: These embeddings are the input to transformer layers for tasks like text generation or classification.

Let me break down the mathematical transformations from Step 4 (Embeddings) → Step 5 (LayerNorm) → Step 6 (QKV Projections) in detail, with clear equations and conceptual explanations.

Step 4: Embedded Input Vectors
Mathematical Representation
– Input: Token IDs [464, 6193, 318] → Embedded via lookup table \( W_e \in \mathbb{R}^{50257 \times 768} \).
– Output: \( E \in \mathbb{R}^{1 \times 3 \times 768} \) (batch_size=1, seq_len=3, hidden_dim=768)
\[
E = \begin{bmatrix}
[-0.0626, -0.0449, 0.0559, \dots] & \text{(Token 1: “The”)} \\
[0.1632, 0.1023, 0.0634, \dots] & \text{(Token 2: “weather”)} \\
[-0.0006, 0.0075, 0.0307, \dots] & \text{(Token 3: “is”)}
\end{bmatrix}
\]

Key Points
✔ Raw embeddings capture initial semantic representations of tokens.
✔ Values are untrained or pre-trained (depending on model initialization).
✔ Next step: Normalization to stabilize training.

Step 5: LayerNorm Output
Mathematical Transformation
Applies Layer Normalization to each token’s embedding independently:
\[
\text{LayerNorm}(E_i) = \gamma \odot \frac{E_i – \mu_i}{\sigma_i + \epsilon} + \beta
\]
where:
– \( E_i \in \mathbb{R}^{768} \): Embedding of the \(i\)-th token.
– \( \mu_i, \sigma_i \): Mean and standard deviation of \( E_i \).
– \( \gamma, \beta \in \mathbb{R}^{768} \): Learnable scale and shift parameters.
– \( \epsilon \approx 10^{-5} \): Small constant for numerical stability.

Example Calculation (Token 1)
1. Compute mean and std:
\[
\mu_1 = \text{mean}([-0.0626, -0.0449, \dots]) \approx -0.042
\]
\[
\sigma_1 = \text{std}([-0.0626, -0.0449, \dots]) \approx 0.078
\]
2. Normalize and scale:
\[
\text{LayerNorm}(E_1) = \gamma \odot \frac{[-0.0626, -0.0449, \dots] + 0.042}{0.078} + \beta
\]
Result:
\[
[-0.1117, -0.0560, 0.0642, \dots]
\]

Why LayerNorm?
✔ Stabilizes training by normalizing activations.
✔ Token-wise normalization (unlike BatchNorm).
✔ Preserves sequence-length independence.

Output
\[
E_{\text{norm}} = \begin{bmatrix}
[-0.1117, -0.0560, 0.0642, \dots] \\
[0.2921, 0.1648, 0.0607, \dots] \\
[0.0089, 0.0305, 0.0303, \dots]
\end{bmatrix}
\]

Step 6: QKV Projections
Mathematical Operations
Projects normalized embeddings into Query (Q), Key (K), Value (V) using learned matrices:
\[
Q = E_{\text{norm}} W_Q, \quad K = E_{\text{norm}} W_K, \quad V = E_{\text{norm}} W_V
\]
where:
– \( W_Q, W_K, W_V \in \mathbb{R}^{768 \times 768} \) (for single-head attention).
– In multi-head attention, these are split into smaller matrices.

Intuition
– Query (Q): “What am I looking for?”
– Key (K): “What information do I contain?”
– Value (V): “What should I output?”

Example Calculation (Token 1)
\[
Q_1 = [-0.1117, -0.0560, \dots] \cdot W_Q = [-0.8680, -1.4208, \dots]
\]
\[
K_1 = [-0.1117, -0.0560, \dots] \cdot W_K = [1.7470, -1.8736, \dots]
\]
\[
V_1 = [-0.1117, -0.0560, \dots] \cdot W_V = [0.0318, -0.0429, \dots]
\]

Output Shapes
All Q, K, V have shape [1, 3, 768] (same as input).

Key Observations
1. Q and K are used for attention scores (dot products).
2. V stores the actual content to be weighted by attention.
3. Projections are linear but enable non-linear interactions via attention.

Description: The normalized embeddings are now projected into Query (Q), Key (K), and Value (V) matrices, which are the foundation of the self-attention mechanism.

Mathematical Formulation

Given an input $X \in \mathbb{R}^{T \times d}$ (where $T = 3, d = 768$):

$$
Q = XW_Q, \quad K = XW_K, \quad V = XW_V
$$

Where:

* $W_Q, W_K, W_V \in \mathbb{R}^{d \times d}$ are learnable weight matrices (shared or per-head in multi-head attention).
* Resulting shapes:

$$
Q, K, V \in \mathbb{R}^{T \times d}
$$

Each token gets projected into three different views:

* Query (what this token wants to attend to)
* Key (how much it should be attended to)
* Value (what content it carries)

These are used in the scaled dot-product attention:

$$
\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^\top}{\sqrt{d}}\right)V
$$

Weight Matrices:
– \( W_Q, W_K, W_V \in \mathbb{R}^{768 \times 768} \)
Projections:
\[
Q = E_{\text{norm}} W_Q, \quad K = E_{\text{norm}} W_K, \quad V = E_{\text{norm}} W_V
\]
Outputs:
\[
Q = \begin{bmatrix}
[-0.8680, -1.4208, \dots] \\
[1.3773, -0.1343, \dots] \\
[0.7626, -1.5240, \dots]
\end{bmatrix}, \quad
K = \begin{bmatrix}
[1.7470, -1.8736, \dots] \\
[1.1331, -0.9781, \dots] \\
[0.4763, -1.3152, \dots]
\end{bmatrix}, \quad
V = \begin{bmatrix}
[0.0318, -0.0429, \dots] \\
[0.3991, 0.1155, \dots] \\
[0.4426, -0.4031, \dots]
\end{bmatrix}
\]

Step 7: Attention Scores
Equation:
\[
\text{Scores} = \frac{QK^T}{\sqrt{d_k}} \quad (d_k = 768)
\]
Computed Scores:
\[
\text{Scores} = \begin{bmatrix}
[21.34, 0.78, 2.95] \\
[1.06, 23.22, -2.82] \\
[5.93, 1.06, 22.16]
\end{bmatrix}
\]

Step 8: Attention Weights (Softmax)
Equation:
\[
\text{Weights} = \text{Softmax}(\text{Scores}) = \begin{bmatrix}
[1.0, 1e-9, 1e-8] \\
[2e-10, 1.0, 5e-12] \\
[9e-8, 7e-10, 1.0]
\end{bmatrix}
\]

Step 9: Attention Output
Equation:
\[
\text{Output} = \text{Weights} \cdot V + E_{\text{norm}}
\]
Result:
\[
\text{Output} = \begin{bmatrix}
[0.8015, -2.0873, \dots] \\
[2.5527, 1.6109, \dots] \\
[-3.7829, 3.6026, \dots]
\end{bmatrix}
\]

Step 10-12: MLP Processing
1. LayerNorm: Normalize attention output.
2. MLP Expansion:
\[
\text{MLP}_{\text{intermediate}} = \text{GELU}(X W_1), \quad W_1 \in \mathbb{R}^{768 \times 3072}
\]
3. Projection Back:
\[
\text{Output} = \text{MLP}_{\text{intermediate}} W_2, \quad W_2 \in \mathbb{R}^{3072 \times 768}
\]

Step 13: Final Transformer Output
Contextual Embeddings:
\[
\text{Output} = \begin{bmatrix}
[-0.0301, 0.3668, \dots] \\
[0.5815, 0.2135, \dots] \\
[0.2250, 0.5953, \dots]
\end{bmatrix}
\]

Step 14-15: Next-Token Prediction
Logits Calculation:
\[
\text{Logits} = \text{Output} W_{\text{lm\_head}}, \quad W_{\text{lm\_head}} \in \mathbb{R}^{768 \times 50257}
\]
Top Predictions for “is”:
\[
P(\text{next token}) = \text{Softmax}(\text{Logits}[-1]) \implies \text{“getting”} (42.4%)
\]

Step 16-18: Autoregressive Generation
Final Output:

Key Mathematical Flow
1. Token → Embedding: \( X \rightarrow E = W_e[X] \)
2. LayerNorm: \( E \rightarrow E_{\text{norm}} \)
3. QKV Projections: \( E_{\text{norm}} \rightarrow Q, K, V \)
4. Attention: \( \text{Softmax}(QK^T/\sqrt{d_k}) \cdot V \)
5. MLP: \( \text{GELU}(X W_1) W_2 \)
6. Prediction: \( \text{Output} W_{\text{lm\_head}} \rightarrow \text{Softmax} \)

All matrices (\( W_e, W_Q, W_K, W_V, W_1, W_2, W_{\text{lm\_head}} \)) are learned during training. This end-to-end process enables transformers to generate coherent text.

Summary steps:

Step 1: Define the Use Case

Identify the purpose of your LLM. Different applications require different designs and datasets.

• General-Purpose LLM: Trained on diverse data for broad tasks (e.g., GPT, BERT).
• Domain-Specific LLM: Focused on specialized fields like legal, medical, or financial text.
• Task-Specific LLM: Designed for tasks such as summarization, translation, or sentiment analysis.

Step 2: Gather and Prepare Data