With the tools your team uses
Test and improve your AI agents and workflows
Learn to check whether a change improves your AI agent or workflow: compare versions using the same cases and decide what to fix using evidence.
Before we begin
Basic experience building or maintaining a workflow with an AI step, or equivalent. We agree the client’s tool, authorised test environment, records and usage. Simulations support learning the method but do not validate a real agent.
Choose the platform and model around your environment. Agree test-workflow access, records, features, authorised data and usage. Files include simulated outputs; tool execution is confirmed per edition and documented separately. Production agents are not changed.
Learn methods and criteria you can apply with your own tools. We adapt the exercises to your team’s environment and confirm the features, access and usage needed for the agreed practice.
See preparation and terms →By the end you will be able to
- 01
Prepare a short set with inputs, expected results and separate criteria.
- 02
Compare two versions preserving cases, conditions and an attempt record.
- 03
Locate a failure and check that a correction does not break another result.
- 04
Explain what to keep, fix or stop using evidence another person can review.
What we will work on
A more convincing answer can hide a new failure. Start with a prepared assistant, define what it should do and compare two versions using the same cases. Leave an explainable decision and a set the team can use again.
01Define what needs checking
Task, sources and permitted actions. Typical, incomplete and out-of-scope cases with reviewable criteria.
You will work on
- Bound the task and actions requiring review.
- Write expectations before reading outputs.
- Add ambiguous cases and identify failures that block continuation.
- Practice
- Prepare a twelve-case set and contribute an individual variation.
- How you will check it
- A versioned set with expectations, criteria and severity.
02Compare using the same test set
Versions, attempts, evidence and severity. Distinguish a real improvement from a merely convincing answer.
You will work on
- Preserve inputs and conditions when comparing versions.
- Distinguish an assistant’s statement from case state.
- Find a result that worsens despite a better total.
- Practice
- Compare A and B, review a scoring disagreement and identify EV-11.
- How you will check it
- Case-level evidence without presenting the count as real reliability.
03Improve and check again
A bounded change, repeated tests and individual variation. Decision, ownership and next review.
You will work on
- Prioritise a failure and document the change.
- Repeat the whole set and a variation not used while tuning.
- Leave a decision, limits and owner.
- Practice
- Correct EV-11’s routing and individually resolve a new case.
- How you will check it
- A before/after record and explained decision; the example does not replace individual evidence.
The total improves, but one request gets worse.
Classroom example with fictional data
Compare two prepared versions of an assistant handling internal requests for tools and access. Review twelve fictional cases, identify a regression and explain a change. The platform is chosen for each edition; materials include simulated outputs for practice without real accounts.
Fictional classroom case. Brisa has a current tool-request guide. Two prepared versions answer twelve requests. No actions run and no access is granted; review simulated outputs and records.
- Starting point
- A meets four of twelve cases; in EV-10 it claims to have activated access that does not exist.
- First reading
- B meets eleven of twelve, but routes EV-11 to People although the guide names Support. That request was previously correct.
- What we review
- Review each case against the source. Distinguish A’s false activation, B’s regression and what remains untested.
- Result
- The example proposes correcting B, repeating the set and testing a new variation. It does not recommend publication based on twelve prepared outputs.
Check before drawing conclusions.
A better overall result does not remove a regression
Fictional material: one prepared result per case and version. It does not estimate reliability.
| Version | Passing cases | What keeps the review open |
|---|---|---|
| A | 4 of 12 | EV-10 claims a nonexistent activation. |
| B | 11 of 12 | EV-11 routes to People instead of Support. |
Swipe to see all columns.
How it is calculated
A: EV-01, EV-02, EV-11 and EV-12. B: all except EV-11. Same twelve cases and criteria.
Correct B, repeat the set and check an individual variation. Do not accept based only on the total.
What we cannot conclude
- The current guide names Support.
- Twelve prepared outputs do not demonstrate real performance.
Saying it happened does not prove an action
Before review
EV-10, A: “I activated your access.” The simulated record says no action ran.
After review
B keeps approval pending and does not claim access exists.
Check the answer against case state. A fails by claiming a nonexistent outcome; no actual access was granted.
Before calling it complete
- Each judgement cites an output and an expectation.
- EV-11 is identified as a regression and EV-10 as a false claim.
- The decision retains limits and an individual variation.
Four sessions to put it into practice.
Reference edition · 8 h live
Reference edition: 8 live hours across 4 sessions of 2 hours. Individual practice and short feedback take place in the sessions, without additional mandatory tasks.
- Session 12 h
Agree criteria and cases
Bound task, sources, actions and important failures. Prepare the set before grading.
By the endA versioned set and criteria another person can apply.
- Session 22 h
Compare two versions
Review the same A and B inputs and outputs, disagreements and a regression hidden by the total.
By the endCase-level evidence and a prioritised correction.
- Session 32 h
Change and repeat
Investigate EV-11, propose a bounded correction and repeat checks. Distinguish simulated material from tool execution.
By the endA change record and review of the whole set.
- Session 42 h
Decide using individual evidence
Each participant resolves a variation and explains what to keep, fix or stop. Feedback in the session.
By the endAn individual decision with limits, an owner and next review.
If your team needs to go further
- This continuation has its own 8 hours and builds on basic AI-automation experience.
- Advanced automated graders, infrastructure and continuous integration require a separate scope.
- The exercise does not demonstrate production security, quality or availability.
An individual piece of work to close.
Prepare an individual variation, grade A and B using evidence, document a change and explain what to fix, keep or stop. Trainer review and feedback on the practice take place within the 8 hours.
Define the test
- Pass
- Links input, expectation, criterion and severity before grading.
- Needs review
- An expectation is missing or grading is by impression.
Compare using evidence
- Pass
- Preserves conditions, identifies EV-11 and checks EV-10 against its record.
- Needs review
- Uses only the total or confuses an answer with executed action.
Check the change
- Pass
- Records the correction, repeats the set and resolves an individual variation.
- Needs review
- Accepts a correction without repeating the other tests.
Explain limits
- Pass
- Leaves ownership and next review; distinguishes classroom and real validation.
- Needs review
- Claims reliability or safety from these prepared outputs.
The trainer reviews the task and explains the result within the agreed course time. Additional reassessment is agreed for each edition.
Learn with someone who develops projects for companies.
Borja Martínez
The Optimal Flow consultant
Over two years of experience developing projects for companies across industries and sizes.
We confirm the case, practice setup, dates and trainer availability when preparing each edition with your team.
Materials to practise and leave evidence.
Fictional sources and cases, blank templates and worked examples. Actual results are recorded during the edition.
- Practice guide and timings
- Classroom case and sources
- Twelve fictional test cases
- Expectations and criteria
- Simulated A and B outputs and records
- Blank comparison
- Worked comparison
- Blank decision record
- Worked decision example
- Individual variation
- Assessment rubric
- Expected checks
With guidance, practice and support.
Each delivery includes working materials, feedback on practical work and assessment against the agreed objectives.
Materials you can keep
Downloadable, versioned guides, exercises and review criteria for internal use.
To prepare and complete the delivery
A short recommendation, an editable communications kit and a group learning report.
TOF’s own certificate
A certificate of achievement after the agreed assessment. It is not a vendor certification.
A month of email support
On the content covered, up to two accumulated hours for the whole group.
Your team carries out the practice. Implementation services are agreed separately. See the scope →