transformers

The usual one-sentence explanation of a language model is that it predicts the next word. That is correct, and on its own it settles nothing, because it skips the two parts that would tell you how the thing behaves. What does a prediction look like before a word comes out of it? And what does…