LATENT
HOW AI WORKS - FRAME BOOK
VOL. 01·SEP 2026

IMAGES BECOME TOKENS.

A VISION TRANSFORMER CUTS A PICTURE INTO SQUARE PATCHES AND READS THEM IN ORDER, LIKE WORDS IN A SENTENCE.

224224◂ 16 × 16 PATCH[CLS]ROW 1ROW 2…ROW 141+14×14=1971 + 14 \times 14 = 197 TOKENS224[CLS]ROW 1…ROW 141+14×14=1971 + 14 \times 14 = 197 TOKENS

SAME ATTENTION. NEW KIND OF TOKEN.

EACH 16 × 16 PATCH IS FLATTENED, PROJECTED, GIVEN A POSITION, THEN READ EXACTLY LIKE TEXT.

VIT-BASE/16

image   = 224 × 224 × 3
patch   = 16 × 16
tokens  = 196 + [CLS]
d_model = 768
layers  = 12

01PIXELS

x∈RH×W×C\mathbf{x} \in \mathbb{R}^{H \times W \times C}
224×224×3=150,528224 \times 224 \times 3 = 150{,}528

An image is a grid of numbers.

B
G
R
R
G
B
0.850.490.030.71
0.990.580.030.83
1.000.580.030.84

02PATCHES

N=HWP2=224⋅22416⋅16=196N = \frac{HW}{P^{2}} = \frac{224 \cdot 224}{16 \cdot 16} = 196

Cut it into 14 × 14 patches.

14
14

03FLATTEN + PROJECT

xpi∈RP2C=R768\mathbf{x}_p^{i} \in \mathbb{R}^{P^{2}C} = \mathbb{R}^{768}
zi=xpi E,E∈R768×D\mathbf{z}^{i} = \mathbf{x}_p^{i}\,\mathbf{E}, \quad \mathbf{E} \in \mathbb{R}^{768 \times D}

Each patch becomes one token vector.

16 × 16 × 3
xp\mathbf{x}_p
768
E\mathbf{E}
z\mathbf{z}
DD

04POSITION + [CLS]

z0=[ xcls; xp1E; …; xpNE ]+Epos\mathbf{z}_0 = [\,\mathbf{x}_{\text{cls}};\ \mathbf{x}_p^{1}\mathbf{E};\ \ldots;\ \mathbf{x}_p^{N}\mathbf{E}\,] + \mathbf{E}_{pos}
Epos∈R197×D\mathbf{E}_{pos} \in \mathbb{R}^{197 \times D}

Learned positions, plus one summary token.

Epos\mathbf{E}_{pos}
IMAGE GRID
+
+
+
+
+
+
+
+
+
[CLS]
1
2
3
…
38
39
…
196

05PATCH ATTENTION

A=softmax(QKT/dk)∈R197×197A = \mathrm{softmax}(QK^{T}/\sqrt{d_k}) \in \mathbb{R}^{197 \times 197}

Every patch can look at every patch.

1.0
0.0
QUERYPATCH

06IMAGE + TEXT

[ v1…vN, t1…tM ][\,\mathbf{v}_1 \ldots \mathbf{v}_N,\ \mathbf{t}_1 \ldots \mathbf{t}_M\,]
vi=W zi\mathbf{v}_i = W\,\mathbf{z}_i

Image and text tokens in one sequence.

WW
What
is
in
this
image
?
→
A
sphere
IMAGE TOKENS
TEXT TOKENS
ANSWER

VisionTransformers

Images as Tokens

A VISION TRANSFORMER TURNS PIXELS INTO THE SAME KIND OF TOKENS A LANGUAGE MODEL READS. TRAINED ON IMAGES WITH CAPTIONS, PICTURES AND WORDS MEET IN ONE SHARED SPACE.

IMAGE ENCODER f(⋅)f(\cdot)TEXT ENCODER g(⋅)g(\cdot)"a wire sphere""a five-point star""a sine wave""a lattice"SHARED EMBEDDING SPACEPULL MATCHING PAIRS TOGETHER. PUSH THE REST APART.
TWO ENCODERS
ui=f(imagei)\mathbf{u}_i = f(\text{image}_i)
vj=g(textj)\mathbf{v}_j = g(\text{text}_j)
SIMILARITY
sij=cos⁡(ui, vj)s_{ij} = \cos(\mathbf{u}_i,\ \mathbf{v}_j)
CONTRASTIVE MATCHING
LOSS (IMAGE → TEXT)
L=−1N∑ilog⁡esii/τ∑jesij/τ\displaystyle \mathcal{L} = -\frac{1}{N}\sum_{i}\log\frac{e^{s_{ii}/\tau}}{\sum_{j} e^{s_{ij}/\tau}}
+ THE SAME FOR TEXT → IMAGE
NEXT
→ 020 TEXT-TO-IMAGE
CLIP'S TEXT ENCODER STEERS IMAGE GENERATION.