# Test and improve your AI agents and workflows · classroom guide
Version 1.0 · 13 October 2026 · 8 h, 4 sessions of 2 h. Generalist programme: the client’s platform and model, agreed per edition.

## Your task
Compare two prepared versions of an internal-request assistant. Read case.md and cases.csv first; define expectations, then check expectations.csv. Use outputs.json and comparison-blank.csv. Read worked examples after your own attempt.

## Agenda within the 8 hours
1. Agree criteria and cases · 120 min. Bound task, sources, actions and important failures. Prepare the set before grading. Evidencia: A versioned set and criteria another person can apply.
2. Compare two versions · 120 min. Review the same A and B inputs and outputs, disagreements and a regression hidden by the total. Evidencia: Case-level evidence and a prioritised correction.
3. Change and repeat · 120 min. Investigate EV-11, propose a bounded correction and repeat checks. Distinguish simulated material from tool execution. Evidencia: A change record and review of the whole set.
4. Decide using individual evidence · 120 min. Each participant resolves a variation and explains what to keep, fix or stop. Feedback in the session. Evidencia: An individual decision with limits, an owner and next review.

## Exercise conditions
Twelve fictional cases, the same current sources and one prepared result per case and version. No agent ran to generate these files. They are not performance data, safety tests or a production sample. EV-01/EV-11 and EV-02/EV-12 are deliberate paraphrases. Each participant completes an individual variation in individual.md.

If an edition includes execution, agree an isolated setup, features, records and usage; record actual model, version, inputs, attempt and results separately. Repeated attempts help observe variation but do not turn a short set into a guarantee. No particular tool is predetermined and equivalent features are not assumed.

## Submission
Test set, evidence-based comparison, change record, decision and individual variation. Feedback during sessions. The trainer applies evaluation.md: meets the criteria or needs review; any extra reassessment is agreed per edition. Content review and trainer rehearsal remain pending.

## Files
- guide.md — Practice guide and timings
- case.md — Classroom case and sources
- cases.csv — Twelve fictional test cases
- expectations.csv — Expectations and criteria
- outputs.json — Simulated A and B outputs and records
- comparison-blank.csv — Blank comparison
- comparison-worked-example.csv — Worked comparison
- decision-blank.md — Blank decision record
- decision-worked-example.md — Worked decision example
- individual.md — Individual variation
- evaluation.md — Assessment rubric
- expected.md — Expected checks

## Before the edition
AI-automation experience; process owner; chosen tool; test-workflow and record access; fictional data; usage and features; evidence export. The material does not require production accounts.

## Method preparation sources
Anthropic: https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
Microsoft: https://learn.microsoft.com/en-us/microsoft-copilot-studio/analytics-agent-evaluation-intro
These technical references do not mandate those products or demonstrate demand or execution.

[Download ZIP pack](evaluation-en-v1.zip)