An interactive explainer

A model does not know what comes next. It scores all 50,000 words and picks one.

One sentence, with the last word missing. Watch the model look back over every word it has, decide which ones matter, and turn that into a word. Attention is the part that decides which ones matter.

LLMs · Attention · Tokens and sampling

AI & Machine Learning · Advanced · 12 min · How an LLM picks the next word — an interactive explainer on ViLearn, with a simulation and a knowledge check. All explainers.

Related: How mixture of experts makes giant models affordable · How gradient descent makes machines learn · Why sending the model more can make the answer worse