Skip to content
Ben Kennedy Explainers · Inside a Transformer
Six stages · One prediction

Context in.
Next word out.

Follow one sentence through a transformer, stage by stage. Tokens become vectors, attention gathers context, and the model ranks every candidate for the next word. Press play, or step through it yourself.

1 of 6
Stage 01 · Goal Predict what should come next
Turning context into probabilities
The big idea

The model is always solving one immediate problem

Given the text it can already see, which token is most likely to come next?

The model never guesses.
It ranks.

Every stage exists to sharpen one ranked list of continuations. Context flows in, probabilities come out, and the top of the list becomes the next word. Then the whole machine runs again.

Ben Kennedy is an AI Engineer and Researcher, PhD in Artificial Intelligence. Founder of Kennedy Applied Sciences Research Group, a self-funded applied AI research practice, and creator of StratoSort.

Want to see the attention step up close? The companion demo, how a transformer reads a sentence, lets you tap any word and watch it look back at earlier words to resolve what comes next.