Tokens are pieces of text.
A token can be a word, part of a word, punctuation, or a space. Models read and write tokens rather than complete answers.
The cat purred.
The model predicts, then repeats.
At each step, it scores possible next tokens. Pick one and it recalculates from there. In the game, likely choices appear larger and brighter.
Generation is not retrieval.
The model usually builds a response from learned patterns rather than retrieving a finished answer. A fluent path can lead to a fact—or plausible nonsense.
High probability means likely, not necessarily true. Low probability can be creative—or wrong.
Play Token Tumble→