LATENT
HOW AI WORKS - FRAME BOOK
VOL. 01·SEP 2026

WHAT LIVESINSIDE THELATENT SPACE?

A MODEL'S HIDDEN STATE IS A DENSE LIST OF NUMBERS. INTERPRETABILITY TRIES TO SPLIT IT INTO FEATURES: DIRECTIONS THAT TRACK CONCEPTS WE CAN NAME.

ACTIVE RESEARCH.READ CLAIMS AS PROVISIONAL.

#0412#5530#0937#4480#7702#1207#3308#2291#6015xxWencW_{\text{enc}}64 OF 16,384FEATURES SHOWN

64 OF 16,384 FEATURES SHOWN

FEATURE PRISM

A SPARSE AUTOENCODER UNFOLDS ONE DENSE ACTIVATION INTO MANY FEATURES. ONLY A FEW LIGHT UP FOR ANY TOKEN.

x≈b+∑i=1mfi(x) dix \approx b + \sum_{i=1}^{m} f_i(x)\, d_i

d = 768m = 16,384active ≈\approx 30 per token

TOKENGateBridgeTheGolden

#1207
golden gate bridge
0.93
#3308
san francisco
0.71
#6015
landmarks
0.54
#1207
golden gate bridge
0.97
#4480
bridges
0.82
#6015
landmarks
0.60
#0412
start of sentence
0.84
#0937
articles (the, a)
0.77
#2291
english text
0.62
#5530
gold / golden
0.88
#7702
capitalized word
0.66
#2291
english text
0.58

01NEURONS

a=ReLU(w⋅x+b)a = \mathrm{ReLU}(w \cdot x + b)
a=ReLU(w⋅x+b)a = \mathrm{ReLU}(w \cdot x + b)

ONE NEURON, SEVERAL UNRELATED MEANINGS.

REPORTED FOR A NEURON IN A SMALL MODEL.

academic citations
English dialogue
HTTP requests
Korean text
aa

02SUPERPOSITION

x^=ReLU(W⊤Wx+b),W∈R2×5\hat{x} = \mathrm{ReLU}(W^{\top} W x + b), \quad W \in \mathbb{R}^{2 \times 5}
x^=ReLU(W⊤Wx+b),W∈R2×5\hat{x} = \mathrm{ReLU}(W^{\top} W x + b), \quad W \in \mathbb{R}^{2 \times 5}

FIVE FEATURES SHARE TWO DIMENSIONS.

WORKS BECAUSE FEATURES ARE RARELY ACTIVE TOGETHER.

h1h_1
h2h_2
n=5, d=2n = 5,\ d = 2

03SPARSE AUTOENCODER

f=ReLU(Wenc x+benc)x^=Wdec f+bdecf = \mathrm{ReLU}(W_{\text{enc}}\, x + b_{\text{enc}}) \qquad \hat{x} = W_{\text{dec}}\, f + b_{\text{dec}}
L=∥x−x^∥22+λ ∥f∥1\mathcal{L} = \lVert x - \hat{x} \rVert_2^2 + \lambda\, \lVert f \rVert_1
f=ReLU(Wenc x+benc)f = \mathrm{ReLU}(W_{\text{enc}}\, x + b_{\text{enc}})
x^=Wdec f+bdec\hat{x} = W_{\text{dec}}\, f + b_{\text{dec}}
L=∥x−x^∥22+λ ∥f∥1\mathcal{L} = \lVert x - \hat{x} \rVert_2^2 + \lambda\, \lVert f \rVert_1

REBUILD x FROM A FEW ACTIVE FEATURES.

d = 768 →\rightarrow m = 16,384 →\rightarrow d = 768 (ILLUSTRATIVE)

xx
WencW_{\text{enc}}
ff
WdecW_{\text{dec}}
x^\hat{x}

04FEATURE DIRECTION

fi(x)=ReLU(di⋅x−θi)f_i(x) = \mathrm{ReLU}(d_i \cdot x - \theta_i)
fi(x)=ReLU(di⋅x−θi)f_i(x) = \mathrm{ReLU}(d_i \cdot x - \theta_i)

A FEATURE IS A DIRECTION IN ACTIVATION SPACE.

PROJECT ONTO IT, THEN CUT AT A THRESHOLD.

did_i
θi\theta_i

05PROBING

p=σ(w⊤x+b)p = \sigma(w^{\top} x + b)
p=σ(w⊤x+b)p = \sigma(w^{\top} x + b)

A LINEAR PROBE TESTS WHAT IS ENCODED.

DECODABLE DOES NOT MEAN USED.

true statements
false statements
ww
p(true) = 0.97

06ATTRIBUTION

Ai=fi⋅∂y∂fiA_i = f_i \cdot \dfrac{\partial y}{\partial f_i}
Ai=fi⋅∂y∂fiA_i = f_i \cdot \dfrac{\partial y}{\partial f_i}

TRACE WHICH FEATURES DROVE THE ANSWER.

A FIRST-ORDER ESTIMATE, NOT A PROOF.

Paris
capital
of
France
capital city
France

07STEERING

x′=x+α dix' = x + \alpha\, d_i
x′=x+α dix' = x + \alpha\, d_i

ADD A FEATURE DIRECTION; BEHAVIOR SHIFTS.

E.G. BOOSTING A "GOLDEN GATE BRIDGE" FEATURE.

"My favorite place is …"
home
beach
Paris
bridge
xx
x′x'
αdi\alpha d_i
α=\alpha =0.0

Interpretability

Features in Latent Space

INTERPRETABILITY ASKS WHAT A TRAINED MODEL REPRESENTS AND HOW IT USES IT. NEURONS MIX MANY CONCEPTS, SO RESEARCHERS LEARN SPARSE DICTIONARIES THAT SPLIT ACTIVATIONS INTO FEATURES THEY CAN NAME, TEST AND STEER.

x′x'xxdid_iαdi\alpha d_iRd\mathbb{R}^dLATENT SPACE
ACTIVATION (RESIDUAL STREAM)
x∈Rdx \in \mathbb{R}^{d}
FEATURE DIRECTION
di∈Rd,  ∥di∥=1d_i \in \mathbb{R}^{d},\ \ \lVert d_i \rVert = 1
SPARSE CODE
f∈R≥0m,  m≫df \in \mathbb{R}_{\ge 0}^{m},\ \ m \gg d
SPARSITY
∥f∥0≪m\lVert f \rVert_0 \ll m
STEERING
x′=x+α dix' = x + \alpha\, d_i

CAVEATS

  • LABELS ARE HUMAN INTERPRETATIONS.
  • DICTIONARIES MISS PART OF THE SIGNAL.
  • DECODABLE IS NOT THE SAME AS USED.