AI as a highly trained text predictor

A large language model (LLM) is the engine behind almost every AI assistant you use for writing. The most honest way to understand a large language model is as a highly trained text predictor. During model training it reads a very large amount of text: books, articles, web pages, documents, conversations. From that reading it does not memorise a list of facts. It learns patterns: after words like these, words like those usually follow.

When you type a question, the model does not open a cabinet of right answers. It reads all the text in front of it, then writes the answer piece by piece. At each step it weighs many possible next pieces and picks one plausible continuation. That piece is added to the text, and the next step starts again, until the answer is finished. These small pieces are called tokens; the lesson on tokens and the context window in this module goes further.

Note the word "one". The model does not always pick the single most likely continuation. It picks from several reasonable ones, which is why the same question can get different answers. The lesson "Why the answers keep changing" explains this.

This way of working is where its strength comes from. The model is very good at imitating form: a formal letter, meeting minutes, a list of steps, a friendly or firm tone, simpler language. All of these are patterns it has seen very often.

The same way of working is where its weakness comes from. Names, figures, dates, article numbers and book titles are also just patterns to the model. If they appeared rarely in its reading, or your question is about something it never read, it does not stop and say "I don't know". It keeps writing a continuation that sounds natural. The result can be a figure that looks precise, a quote that sounds official or a reference that looks convincing, with no basis at all. A fluent, confident answer like this is called a hallucination. Fluency is the model's skill; truth is a separate thing to check.

One more thing makes fluent answers risky: the tone is almost always equally confident, whether the answer is right or invented. A person who is unsure usually sounds unsure. A model does not. It can write a wrong sentence with the same tidy wording, spelling and tone as a right one. So tidy, long or full of jargon is no sign that an answer has been checked.

That is why a language model is most useful for working on material you give it: summarising, rewriting, structuring, changing shape, finding patterns in text you paste. There the material is in front of the model, and you can compare the result with the source. It is riskiest when asked to recall specific facts without a source: figures, names, dates and rule references. The maternity-leave example, with an invented condition and an outdated rule, is covered in full in the lesson Why models make things up.