Split and merge heads

Follow one number through splitting, transposing, and merging heads. Every head keeps every position; all 12 values stay unchanged.

Current shape
[1, 3, 4]
12 → 12
B = 0
Open the static split diagram

3D is unavailable on this device. The index table below still supports every step.

Preparing the 3D view…

Step

1 / 5
Index table & notes (scroll)

Select a number to inspect its index

This lab only splits, transposes, and merges. It does not compute projections or attention. In full attention, Q/K/V are projected before splitting.

After splitting, the spatial directions represent position, head, and within-head coordinate. Their order changes by step; batch B=1 is fixed. Block thickness only separates cells, not an extra dimension.

Values come from the saved lesson 12 experiment. No training runs here. Open the static split diagram ↗