AI / ML VisualizerRuns in your browser · No API keyOpen Model
← All topics

Advanced

Attention

Inspect QKᵀ, scaled softmax and weighted value vectors.

Scaled dot-product attentionScaled dot-product attentionQ: TheK: The · 0.3701Q: catK: cat · 0.2599Q: satK: sat · 0.3701
QKᵀ / √d · 3 × 3
Attention weights (row softmax) · 3 × 3
Weighted values · 3 × 2
Attention(Q,K,V) = softmax(QKᵀ / √dₖ)VQuery 1 output = [0.5551, 0.4449]

Continue exploring

No account required. Local images and model files stay in your browser. Educational models demonstrate mechanisms; they are not production predictors.