Build a Small Model, Understand Large Models / One actual forward pass
Model Inspector
Follow one character through every layer. Inspect how its 64 values change and where the high score for “红” (red) comes from in the same saved trace.
Saved pretrained weights · 92,352 parameters · Inspecting a trace with fixed parameters
Select a position to followThe full sentence is given. Inspect one forward pass, not a generation loop.
NegativePositive
The 8×8 layout shows one 64-dimensional vector at one position. Color intensity scales to the current vector; the scale changes between layers.
Normalize over all 79 candidates; the top 5 probabilities are shown below.
64 contributions to this candidate
Positive contributions are above zero, negative contributions below. Select a bar to inspect the multiplication.