Module 1 · Start safely
Code from AI is a draft, and you own it
In short
AI code enters the repository under your name. Read its diff, ask for runnable proof, check APIs and library versions against the docs or code, and never accept a done claim without a test you saw yourself.
After this lesson you can
- Explain why code from an assistant or agent stays the responsibility of the developer who merges it
- Read an AI diff with four questions: what changed, why, what proves it and what was left untouched
- Recognise signs of model-invented code: APIs that do not exist, wrong library versions and done claims without proof
Terms in this lesson
Understand
Coding assistants and coding agents can write hundreds of lines in a minute. That speed is real, and it is also the source of the risk: the more code arrives, the stronger the urge to merge it because it looks neat and the tests are green.
Treat every AI output as a draft from a new colleague who is very fast, very confident and has never seen your system run. A colleague like that is useful, but you would not merge their work without reading it. A model does not know your team's conventions unless you tell it. It does not know production data has empty values in a column the schema says cannot be empty. It does not know the function it calls was removed in the library version you use.
That is why a model can make things up. In code, invention takes a few common forms: library functions or options that sound reasonable but do not exist; API use from an older version or a newer one than yours; tests that check what the code does, not what was asked; and a report of "done, all tests pass" when the tests were never run, or were run on the wrong file.
Reading an AI diff takes four questions. What changed? Look at the file list first; an unexpected file is the first warning. Why? Every change must tie back to the task. What proves it? A test that failed before the change and passes after it, or a command you can run yourself. What was left untouched? Sometimes what is missing matters more: an access check not added, a migration not written, or error handling quietly removed.
Your responsibility is not to write every line yourself. It is to make sure you can explain every line that goes in.
Version checks deserve special attention. A model learned from code across many years, so it mixes styles from different library versions. Before accepting code that uses an outside library, open your project's dependency list and see the installed version, then match the function calls with that version's documentation or with the library's source in the dependency folder. If the library has types, the type checker will catch some inventions; the rest only show when the code actually runs.
One small habit helps: ask for small changes. A two-hundred-line diff can be read carefully in ten minutes; a two-thousand-line diff usually gets a glance. If the task is large, split it into steps that each have their own tests, and review each step before moving on. Coding agents work better with limits like that, and you keep understanding your own system.
Before
The prompt
Add a report export to CSV. Make sure it all works.
What came back
Bad example output, do not use: Done. I added an export endpoint and used the toCsv function from the table library. All tests pass.
After
The prompt
Goal: I need a monthly report export to CSV from the reports page. Reader: me, reviewing the diff before merge. Material: the report data function and the library versions in the dependency list. Limits: touch only the reports module and the export route; add no new library; export only for users allowed to see that report. Shape: write the test first and show it failing, then the code, then the passing result. End with the list of changed files. Check: name any library function you use with its version, and write the test command you ran.
What came back
Tests: export rejects users without view rights (fails: route missing), export produces a header and two sample rows (fails). Code: export route with an access check, CSV written by hand with quote escaping. Tests: two pass. Files: export route, reports module, one test file. No new library.
What changed
- The first report names a library function that may not exist and claims passing tests without a command or result.
- The second request limits the files, forbids new libraries and asks for proof of fail then pass.
- The access check appears because Limits asked for it; without it, the export could open the report to anyone.
Common mistake
- Mistake
- Merging a large agent change because its report says all tests pass.
- Why it hurts
- An agent's report is a claim, not evidence. The tests may not have run, may have run on other files, or may have been weakened to pass. The fault shows in production, and by then it is your name in the history.
- Do this instead
- Run the named test command yourself, read the diff of the test files too, and reject changes that alter old tests without a written reason.
Try
Open each card: what is the sign of invented code, and how do you check it?
Search for its name in the library source in the dependency folder or in your version's docs. Not found means invented.
Briefly disable the new code and run again. A test that stays green does not check your change.
Ask why. Changes that cannot be explained are reverted.
Ask for the exact command and run it yourself.
An agent finishes a task and writes: all tests pass. Its diff also changes an old test from must reject to may accept. What is your step?
Take home
Before approving AI code
- I read the list of changed files and know why each one changed
- I ran the test command myself
- I saw the new test fail without the change and pass with it
- I checked unfamiliar library functions against the installed version
- No old test was weakened without a written reason
- I can explain every line I approve
What went wrong for us
A feature claim the code did not back
- Context
- We were writing course material with AI help, split into several work lanes, and every lane was checked by a reviewer separate from its author.
- What happened
- Some drafts described apps we run as if they did things they do not do. The sentences were neat and convincing. The reviewer opened each app's code, looked for the claimed behaviour and found it missing or different.
- Impact
- No reader saw it, because the draft was not yet published. Had it passed, readers would have trusted a capability that does not exist.
- Root cause
- The author inferred behaviour from a feature's name and plan, not from running code. The model filled that gap with a plausible sentence.
- What we did
- The claims were rewritten to match what the code really does, and the other product claims in the same drafts were checked against the code.
- The guard we keep now
- In our course material, every claim about our products is checked against the code before it ships, and reviewers read the code, not the author's summary.
The lessonA convincing summary is not evidence; the code is.
What it means for your workBefore writing that a system does something, open its code or run it, then write only what you saw.