Training and the knowledge cutoff

The model you use in a chat app is shaped in two big stages. Knowing both helps you guess when the model can be relied on and when it cannot.

The first stage is pre-training. The model reads a very large amount of text and learns one thing: continuing text. The result is good at carrying on a sentence but does not necessarily answer questions. If you write a question, a raw model might continue it with another question, as if copying a list of exam items.

The second stage is post-training. First the model is trained further on many examples of good conversations: questions and answers that are helpful, polite and follow instructions. Training an existing model further on specific examples like this is called fine-tuning. Then raters compare several of the model's answers and pick the better one, and the model is tuned to write the answers raters prefer more often. The raters are people, or another model trained to imitate people's judgements. This stage is what makes the model behave as an assistant: answering, tidying up, offering help.

The second stage also has a side effect you should know about. Human raters tend to like answers that are friendly, confident and in line with the person asking. As a result the model tends to agree with you, praise your plan and prefer giving an answer to saying "I don't know". The model agreeing is not proof that you are right. How to ask a model to show the weak spots in your plan is covered in the lesson "A fast new colleague" in the module Ask clearly.

Training sets one more limit: the knowledge cutoff. Text for training is collected up to a certain date. After that the model is trained, tested, released and then used for months, sometimes more than a year. Whatever happened after that date, the model does not know, unless the app fetches new sources or you provide them. Even events shortly before the cutoff are often only partly known, because when the data was collected few people had written about them yet.

How do you know a model's answer is out of date? The model rarely signals it. It will say "the latest version", "the rule in force" or "the current price" in the same tone, when what it means is the state of things when its data was collected. Models also often do not know exactly when their own cutoff is, and do not know today's date unless the app tells them. For regulations, this can mean the model quotes an old rule that was once right, while a newer rule now governs the matter differently; the example is in the lesson Why models make things up.

Suspect a stale answer whenever your question is about something that changes often: regulations and rates, prices and exchange rates, app versions and their features, who holds which post, schedules and recent events. Suspect everything about your own office too: internal policies, product names, sales figures. Those were never in the training data, however new the model is.

Some apps can search the web or read office documents before answering. That helps, but still check the sources it pulls in; the lesson "Research with sources" shows how. The surest way stays simple: paste in the latest material you trust yourself, then ask the model to work only from it.