Kamus AI
Evaluation
A measured way to judge whether AI output is good enough for its task.
Rather than judging from one or two answers that look good, prepare a set of real examples with their expected results, then measure how often the AI is right. Repeat whenever the prompt, model or data changes. A good evaluation includes hard cases and cases that failed before.
Also called: Evaluation, eval, evals, output testing
Related terms
Appears in 16 lessons
- Improving the result over a few rounds
- Checking the output before you use it
- Our review routine
- Making a prototype to test an idea
- Choosing your next level
- Prompts for SOPs and handovers
- Mini project: a safe operations kit
- Prompts for pricing, stock and promotions
- Case: running and closing the season
- Prompts for assessment and feedback
- One teaching unit from start to reflection
- Reading an agent's trace
- Project: a ten-task test set for your agent
- Writing a tested skill
- Fix loops driven by tests
- Reviewers that push back