Module 1 · Start safely
Invented provisions: never cite without opening the text
In short
A model guesses text, so article and judgment numbers are easy to invent. Use a legal reference from AI only after you open the official text and match its wording. Give each reference a status: matches, differs or not found.
After this lesson you can
- Explain why a model can invent article numbers, article wording and judgment numbers that sound convincing
- Check every legal reference from AI against the official text before it enters any document
- Ask AI to work only from legal text you paste and to flag what that text does not contain
Terms in this lesson
Understand
In Level 1 you learned why models make things up: a large language model (LLM) predicts the most likely text; it does not fetch text from a source. In legal work this symptom, usually called hallucination, takes a particular and dangerous form, because legal references follow a pattern that is very easy to imitate: "Article so-and-so paragraph (so-and-so) letter so-and-so", "Judgment Number so-and-so/Pdt.G/year/District Court city". The model has read thousands of such references, so it can produce new ones that are perfectly shaped but never existed.
Inventions in legal work usually come in five forms. The wrong article number: the content is roughly right but sits in another article. Blended wording: a sentence combining two provisions, or a provision from another law. An article that has changed: old wording that has been amended, repealed or declared no longer in force. A fictitious judgment: a number, date and court reasoning that never existed, or a real number with the wrong content. An invented expert quote: a real author's name with a sentence they never wrote.
What makes this dangerous is the confident tone. An answer that names an article sounds more trustworthy than one that names nothing. Yet that number can be invented as easily as any other sentence. From the answer text alone you cannot tell a real reference from an invented one. The only way is to open the source.
So the rule in this track is simple and has no exceptions: no legal reference enters any document, whether an internal memo, a draft contract or a letter, before someone has opened the official text and matched its wording. Official text means regulations from a government regulation database such as JDIH, or a copy of the judgment from the court's official directory, not a summary on a blog or the model's memory. And the model is never its own checker.
To make this check traceable, give every reference one of three statuses. Checked, matches: you opened the official text, the version in force for the facts and dates, and the wording is the same. Checked, differs: the article exists but its wording or number is different; write the correct one and drop the wrong one. Not found: you could not find it in an official source; the reference may not be used until it is found, and if it never is, note that the AI invented it. Also note where and when you opened the text, so the advocate reading your memo can repeat the check in a minute.
Before
The prompt
Name the PDP Law article that sets fines for a company whose data leaks, with its wording.
What came back
Under Article 99(3) of the PDP Law, "Every company negligent in protecting personal data shall be fined at most ten times its annual turnover."
After
The prompt
Goal: I am preparing an internal note on sanctions in the PDP Law. Reader: an advocate on my team, who will check it. Material: the text of the sanctions chapter of Law No. 27 of 2022, copied from the official source, below. Limits: use only the text I paste. Do not add articles from memory. If something is not covered in this text, write "not in the pasted text". Shape: a table with columns article, type of sanction, quote of the original sentence. Check: every row must have a quote I can find with a word search in the text. [sanctions chapter text]
What came back
A table of articles from the pasted text, each with a quote of its original sentence. Last row: "Fine amount for the asked type of breach: not in the pasted text; check other parts of the law or its implementing regulations."
What changed
- The article number and quote in the first answer are invented on purpose for practice; do not use them. The official text of the PDP Law ends well before that number.
- The second request supplies the official text and asks for quotes you can search, so every row can be checked.
- "Not in the pasted text" is more useful than a confident wrong answer.
Common mistake
- Mistake
- Asking the same model, "Is that article correct?", then treating a "yes" as a check.
- Why it hurts
- A model tends to defend its own answer and can invent a confirmation as easily as an article. You gain no new evidence, only a second guess that sounds more confident.
- Do this instead
- Open the official text yourself, find the sentence with a word search, match the number and wording, then record the status, source and date you opened it.
Try
Open each card: what should you do when you see this sign in an AI answer?
Look the article up in the official text. A neat reference is not evidence; its wording at the source decides.
Search the official judgment directory. If it is not found, mark it not found and do not use it.
Ask for the book title and page, then check the book. A real name does not make the sentence real.
Ask which version was used, then check whether that version applies to the date of the events in your matter.
A memo for the advocate is due in an hour. AI gave you three article references. You matched two with the official text; you have not found the third yet. What do you do?
Take home
Check every legal reference from AI
- I open the official text from a regulation database or judgment directory, not a summary
- I find the sentence with a word search and match its number and wording
- I make sure the text version applies to the facts and dates of this matter
- I give a status: checked matches, checked differs, or not found
- I record the source and the date I opened it
- A reference not yet found does not enter the document as a basis
What went wrong for us
Articles that looked right
- Context
- We built an analysis aid that screens social media posts against relevant statutory articles.
- What happened
- In early tests the model cited article numbers and wording that sounded convincing. From the output alone, we could not tell which were right.
- Impact
- Nothing reached users. Had it slipped through, legal analysis could have rested on the wrong articles.
- Root cause
- The model predicts the most likely text, it does not fetch text from a source. Article numbers are a pattern that is easy to imitate.
- What we did
- The tool was changed to show article text from a stored source, not from the model's memory, and to mark all output as a draft.
- The guard we keep now
- Every cited article must be verified by counsel against the official source before use.
The lessonA reference that sounds right still has to be checked at its source.
What it means for your workUse any article number, figure or quote from AI only after you have found it yourself in the official text.