Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Foundations of Mastra Debugging and Evaluation
- Understanding agent behavior models and failure modes
- Core debugging principles within Mastra
- Evaluating deterministic and non-deterministic agent actions
Setting Up Environments for Agent Testing
- Configuring test sandboxes and isolated evaluation spaces
- Capturing logs, traces, and telemetry for detailed analysis
- Preparing datasets and prompts for structured testing
Debugging AI Agent Behavior
- Tracing decision paths and internal reasoning signals
- Identifying hallucinations, errors, and unintended behaviors
- Using observability dashboards for root-cause investigation
Evaluation Metrics and Benchmarking Frameworks
- Defining quantitative and qualitative evaluation metrics
- Measuring accuracy, consistency, and contextual compliance
- Applying benchmark datasets for repeatable assessment
Reliability Engineering for AI Agents
- Designing reliability tests for long-running agents
- Detecting drift and degradation in agent performance
- Implementing safeguards for critical workflows
Quality Assurance Processes and Automation
- Building QA pipelines for continuous evaluation
- Automating regression tests for agent updates
- Integrating QA with CI/CD and enterprise workflows
Advanced Techniques for Hallucination Reduction
- Prompting strategies to reduce undesired outputs
- Validation loops and self-check mechanisms
- Experimenting with model combinations to improve reliability
Reporting, Monitoring, and Continuous Improvement
- Developing QA reports and agent scorecards
- Monitoring long-term behavior and error patterns
- Iterating on evaluation frameworks for evolving systems
Summary and Next Steps
Requirements
- An understanding of AI agent behavior and model interactions
- Experience with debugging or testing complex software systems
- Familiarity with observability or logging tools
Audience
- QA engineers
- AI reliability engineers
- Developers responsible for agent quality and performance
21 Hours
Delivery Options
Private Group Training
Our identity is rooted in delivering exactly what our clients need.
- Pre-course call with your trainer
- Customisation of the learning experience to achieve your goals -
- Bespoke outlines
- Practical hands-on exercises containing data / scenarios recognisable to the learners
- Training scheduled on a date of your choice
- Delivered online, onsite/classroom or hybrid by experts sharing real world experience
Private Group Prices RRP from £5700 online delivery, based on a group of 2 delegates, £1800 per additional delegate (excludes any certification / exam costs). We recommend a maximum group size of 12 for most learning events.
Contact us for an exact quote and to hear our latest promotions
Public Training
Please see our public courses
