一个 token 能读到谁？Who can a token read?
因果遮罩（causal mask）决定哪些位置能进入注意力输出。在一个每个数都算出来的小模块里追踪一个 query，再关掉遮罩做对比。
A causal mask changes which positions contribute to an attention output. Trace one query through a tiny, fully calculated block. Then remove the mask and compare.
01预测Predict
02动手试Try a data flow
在左侧选择 query token，然后开、关遮罩，比较它收到的注意力权重。线条粗细和矩阵颜色都是注意力权重，不是词的概率，也不代表模型“理解”了语言。
Select a query token on the left. Turn the mask on and off and compare the weights it receives. Line width and matrix shading are attention weights, not word probabilities or explanations of language understanding.
用左侧的 7 个阶段按钮（或“上一步 / 下一步”）逐步查看计算过程。Step through the calculation with the seven stage buttons on the left, or with Previous and Next.
查看所有输入和玩具参数Inspect every input and toy parameter
| Token | 嵌入 EEmbedding E | 位置 PPosition P | X = E + P |
|---|
d_model = 4; one head; d_k = d_v = 2. W_Q = W_K = [[1,0],[0,1],[0,0],[0,0]] W_V = [[0,0],[0,0],[1,0],[0,1]] W_O = [[1,0,0,0],[0,0,1,0]] FFN(y) = 0.5 × ReLU(y). W₁ = I₄, W₂ = 0.5 I₄, b₁ = b₂ = 0. LayerNorm uses population variance, ε = 0.00001, γ = 1, β = 0.
03解释Explain
为什么贡献变了？Why did the contribution change?
遮罩在 softmax 之前把被禁止的分数换成负无穷。它们的指数是 0，允许的权重重新归一化，总和仍为 1。换一个 query 会改变分数，但绕不过遮罩。
每个权重乘以一个 value 向量。加权后的 value 相加，再投影回四个坐标。残差路径把输入绕过注意力直接加回来，随后是归一化和逐位置的前馈计算。
The mask replaces forbidden scores with negative infinity before softmax. Their exponentials are zero. Allowed weights renormalize to sum to one. Changing the query changes scores, but it cannot bypass the mask.
Each weight multiplies a value vector. These weighted values are added, then projected back to four coordinates. The residual path carries the input around attention. Normalization and a position-wise feed-forward calculation follow.
“遮罩 ___ 时，___ 的权重是 ___，因为 ___。”“With the mask ___, the weight of ___ was ___ because ___.”
迁移：换一个新情形Transfer
这个例子模拟了什么What this example models
这是一个解码器式、后归一化（post-norm）的模块，只有一个注意力头。页面上的每个数都由上面的玩具参数算出。嵌入、位置向量和矩阵都是手选的：没有训练，也不预测下一个 token。关掉遮罩会让未来信息进入，破坏因果解码。
完整架构是：token 嵌入加位置信息 → 重复的注意力与前馈模块 → 词表投影和 softmax。本页只计算其中一个模块，省略了多头并行、dropout、交叉注意力和词表输出。这里简单的位置向量也不是论文中的正弦位置编码。
注意力、遮罩、残差归一化和 FFN 结构的来源：Vaswani et al., Attention Is All You Need, §3。原论文是带交叉注意力的编码器—解码器架构；这里是简化的解码器式教学模型，不是对整个模型的复现。
This is one decoder-style, post-normalization block with a single attention head. Every displayed number comes from the toy parameters above. Embeddings, position vectors and matrices are hand chosen. Nothing is trained, and no next token is predicted. Turning off the mask permits future information and breaks causal decoding.
The architecture context is token embeddings plus position information → repeated attention and feed-forward blocks → a vocabulary projection and softmax. This page calculates only one block. It omits multi-head parallelism, dropout, cross-attention and vocabulary output. Its simple position vectors are not the paper's sinusoidal encoding.
Source for attention, masking, residual normalization and FFN structure. Vaswani et al., Attention Is All You Need, §3. The original paper uses an encoder-decoder architecture with cross-attention. This toy is a reduced decoder-style teaching model, not a reproduction of that whole model.
离线可用。原生按钮和复选框支持键盘与触控。阶段切换是即时的，没有动画或后台计算。重置会清空你的写作和两道题的答案。Offline. Native buttons and checkbox support keyboard and touch. Stages change instantly; no animation or background work runs. Reset clears your writing and both answers.
选择 query tokenChoose the query token
计算阶段Calculation stages