How Large Language Models Actually Think: Deconstructing the Transformer Brain

Introduction: The Ghost in the Matrix

Every time you type a prompt into an LLM and watch coherent, contextually-aware text spill out, it's tempting to imagine a little homunculus inside the model, deliberating, reasoning, understanding. It isn't. What's actually happening is a staggeringly elegant piece of applied linear algebra — billions of matrix multiplications, softmax normalizations, and residual additions, orchestrated by an architecture called the Transformer. There is no 'thinking' in the human sense. There is attention, there is gradient descent, and there is an almost incomprehensibly large space of learned parameters that somehow, when composed correctly, produce something that looks like thought.

This article deconstructs the Transformer from first principles — token to embedding, embedding to attention, attention to output logits — so you walk away with a mental model precise enough to implement one yourself. We're going deep into the machinery: the matrices, the math, the emergent behavior. Strap in.

Tokens & Embeddings: The Atoms of Thought

Before any 'thinking' happens, raw text has to be converted into something a neural network can consume: numbers. This process begins with tokenization — splitting text into subword units using algorithms like Byte-Pair Encoding (BPE) or SentencePiece. The word 'transformers' might become ['transform', 'ers'], and rare words get decomposed into smaller, more frequent fragments.

Each token is mapped to an integer ID via a vocabulary lookup table, and each ID indexes into an embedding matrix E ∈ R^(V × d), where V is vocabulary size (often 30K–100K+) and d is the model's hidden dimension (512, 768, 4096, or larger depending on model scale). This embedding is not just a static lookup — it's a learned representation that encodes semantic relationships in high-dimensional space.

import torch
import torch.nn as nn

vocab_size = 50257  # GPT-2 style vocabulary
d_model = 768

# The embedding matrix IS the model's 'vocabulary of meaning'
embedding = nn.Embedding(vocab_size, d_model)

token_ids = torch.tensor([[464, 4906, 318, 257, 47686]])  # 'The cat is a robot'
x = embedding(token_ids)  # shape: [1, 5, 768]

print(x.shape)  # torch.Size([1, 5, 768])

Self-Attention: The Mechanism of 'Focus'

Self-attention is the beating heart of the Transformer. For every token in a sequence, the model computes a weighted average of every other token's representation, where the weights are determined by learned relevance scores. This lets the model dynamically decide: 'When processing the word it, how much should I attend to the dog versus the ball?'

Mechanically, each token's embedding is projected into three vectors: Query (Q), Key (K), and Value (V), via learned weight matrices W_Q, W_K, W_V. The Query represents 'what am I looking for,' the Key represents 'what do I contain,' and the Value represents 'what information do I actually carry.'

Here, Q K^T computes a similarity score between every query and every key (a dot product measures alignment in vector space). Dividing by sqrt(d_k) prevents the dot products from growing too large in high dimensions, which would push the softmax into regions with vanishing gradients. The softmax converts raw scores into a probability distribution — the 'attention weights.' Finally, these weights are used to compute a weighted sum of the Value vectors.

Multi-Head Attention: Parallel Perspectives

A single attention computation only captures one 'type' of relationship. Multi-head attention runs several attention operations in parallel, each with its own learned W_Q, W_K, W_V matrices, allowing the model to simultaneously track different relational patterns — one head might specialize in syntactic dependencies (subject-verb agreement), another in coreference resolution (pronoun-antecedent linking), and another in long-range thematic consistency.

class MultiHeadAttention(nn.Module):
    def __init__(self, d_model, num_heads):
        super().__init__()
        assert d_model % num_heads == 0
        self.d_k = d_model // num_heads
        self.num_heads = num_heads

        self.W_q = nn.Linear(d_model, d_model)
        self.W_k = nn.Linear(d_model, d_model)
        self.W_v = nn.Linear(d_model, d_model)
        self.W_o = nn.Linear(d_model, d_model)

    def split_heads(self, x, batch_size):
        x = x.view(batch_size, -1, self.num_heads, self.d_k)
        return x.transpose(1, 2)  # [batch, heads, seq_len, d_k]

    def forward(self, x, mask=None):
        batch_size = x.size(0)
        Q = self.split_heads(self.W_q(x), batch_size)
        K = self.split_heads(self.W_k(x), batch_size)
        V = self.split_heads(self.W_v(x), batch_size)

        scores = torch.matmul(Q, K.transpose(-2, -1)) / (self.d_k ** 0.5)
        if mask is not None:
            scores = scores.masked_fill(mask == 0, float('-inf'))

        attn = torch.softmax(scores, dim=-1)
        out = torch.matmul(attn, V)

        out = out.transpose(1, 2).contiguous().view(batch_size, -1, self.num_heads * self.d_k)
        return self.W_o(out)

GPT-3 uses 96 attention heads per layer across 96 layers. Each head is small individually (128 dimensions in a 12,288-dimensional model) but their concatenated outputs, remixed by W_o, create a remarkably rich representational space.

Positional Encoding: Teaching Order to a Timeless Machine

Here's a subtle but critical problem: self-attention, as described, is permutation invariant. It has no inherent notion of sequence order — 'the dog bit the man' and 'the man bit the dog' would produce identical attention patterns without additional information. Transformers solve this by injecting positional encodings directly into the input embeddings.

The original Transformer paper (Vaswani et al., 2017, 'Attention Is All You Need') used fixed sinusoidal functions:

Modern architectures (LLaMA, GPT-NeoX, and others) have largely moved to Rotary Positional Embeddings (RoPE), which rotate query and key vectors in embedding space by an angle proportional to their position, encoding relative position information directly into the attention dot-product rather than adding it to the input. This tends to generalize better to longer sequences than the model saw during training.

Feed-Forward Networks: Where Knowledge Actually Lives

It's easy to get attention-obsessed and forget that roughly two-thirds of a Transformer's parameters live in the position-wise feed-forward network (FFN) that follows attention in every layer. This is a simple two-layer MLP applied independently to each token position:

The activation function is typically GELU or SwiGLU in modern architectures. The inner dimension is usually 4x the model dimension (e.g., 3072 for a 768-dim model), creating a bottleneck-expand-bottleneck structure. Recent interpretability research (notably from Anthropic's mechanistic interpretability team) suggests these FFN layers function like key-value memory stores — each neuron potentially activating for specific learned concepts, facts, or patterns, sometimes in a highly polysemantic, superimposed way (a phenomenon called superposition).

Stacking Layers: The Emergence of Abstraction

A single attention + FFN block is powerful but limited. The real magic emerges from stacking dozens or hundreds of these blocks, connected via residual connections and stabilized with layer normalization.

Residual connections (the 'x +' part) are crucial — they allow gradients to flow directly through the network during backpropagation without vanishing, enabling stable training of extremely deep networks (GPT-3 has 96 layers; some frontier models exceed 100). Without residuals, deep Transformers simply fail to train.

Interpretability research reveals a rough hierarchy of abstraction across layers: early layers tend to capture surface-level syntactic features (part-of-speech, local dependencies), middle layers capture semantic and relational structure, and later layers capture task-specific and highly abstract representations relevant to the final prediction. This isn't a strict rule — it's an emergent tendency observed through probing classifiers and activation analysis, not a designed feature.

Next-Token Prediction: The Illusion of Reasoning

After passing through every layer, the final hidden state for each position is projected back into vocabulary space via a linear layer (often tied to the input embedding matrix), producing raw logits — one score per vocabulary token. A softmax converts these into a probability distribution, and the model samples (or greedily picks, or uses beam search, or nucleus/top-p sampling) the next token.

This is the crucial, almost uncomfortable truth: an LLM does not 'decide' an entire sentence or plan an argument in advance. It predicts one token at a time, conditioned on everything before it, repeated autoregressively. Apparent 'reasoning' emerges because the training objective — predicting the next token across trillions of tokens of human-generated text — forces the model to internalize statistical regularities so deep and so compressed that they end up encoding grammar, facts, logical patterns, and even rough approximations of multi-step reasoning, purely as a side effect of compression.

To predict the next word well, you eventually have to model the world that generated the words.

Paraphrased sentiment common in interpretability research on emergent capabilities

Building Your Own Transformer Brain

Reading about attention heads and residual streams is one thing. Actually implementing tokenization, embeddings, multi-head attention, RoPE, and a full training loop from scratch is where the real understanding clicks into place. If you want to go beyond theory and build a working, trainable LLM with your own hands — architecture, training loop, sampling strategies, the whole stack — we've put together a complete hands-on build guide.

Head over to https://www.mytruenerd.com/build-an-llm for the full walkthrough on constructing a Transformer-based LLM from the ground up, including code you can run, modify, and break in all the most educational ways.

Conclusion: The Machine That Predicts, Not Thinks

When you strip away the mystique, a Transformer is an extraordinarily well-engineered function approximator: tokens become vectors, attention mixes information across positions using learned relevance scores, feed-forward layers transform that information, residual connections keep gradients flowing through dozens of stacked layers, and the whole system is trained to minimize the error between its predicted next token and reality across a mind-boggling volume of text.

There is no homunculus, no inner monologue, no genuine understanding in the human sense — and yet, from this relatively simple recipe, scaled to billions of parameters and trained on internet-scale corpora, something emerges that can write code, explain quantum mechanics, and hold a conversation about its own architecture. Whether that constitutes 'thinking' is a question for philosophers. For engineers, what matters is that we now understand, with mathematical precision, exactly how the gears turn — and that understanding is the first step toward building something even better.