ISTQB Certified Tester: AI Testing

Course Description

The ISTQB Certified Tester AI Testing is a follow on to the ISTQB Certified Tester: Foundation Level and is valuable to a wide range of professionals involved in the development, testing, and management of AI-based systems with a significant focus on Machine Learning. This certification helps professionals stay updated with the latest AI testing methodologies, improve their skills, and enhance their career opportunities in the rapidly evolving field of AI. This course provides comprehensive training for the ISTQB Certified Tester AI Testing certification. It covers fundamental concepts of AI, significantly focussing on Machine Learning and also, quality characteristics specific to AI-based systems, and various testing techniques and methods applicable to AI-based systems. The course contents will include detailed explanations, practical exercises, and hands-on activities for each topic covered in the course outline.
4
€2400.00
 

Prerequisites

Attendees intending to take the ISTQB Certified Tester AI Testing examination must hold the ISTQB Certified Tester Foundation Level certificate.
To effectively participate in the technical exercises, delegates should set themselves up in advance of the course, on Google Colab or a similar platform as this will enhance the learning experience and provide the necessary tools to complete hands-on exercises.

Course objectives

By the end of this course, participants will be able to:
Understand the fundamental concepts of AI and machine learning.
Identify and explain quality characteristics specific to AI-based systems.
Apply various testing techniques and methods to AI-based systems.
Evaluate and improve the performance of AI models.
Address challenges in testing AI-based systems, including bias, transparency, and explainability.

Who should attend?

This course is designed for:
Testers, test analysts, data analysts, test engineers, test consultants, test managers, user acceptance testers, and software developers involved in testing AI-based systems and/or AI for testing.
Project managers, quality managers, software development managers, business analysts, operations team members, IT directors, and management consultants seeking a basic understanding of testing AI-based systems and/or AI for testing.

Introduction to AI

Definition of AI and AI Effect
Narrow, General, and Super AI
AI-Based and Conventional Systems
AI Technologies
AI Development Frameworks
Hardware for AI-Based Systems
AI as a Service (AIaaS)
Pre-Trained Models
Standards, Regulations, and AI

Quality Characteristics for AI-Based Systems

Flexibility and Adaptability
Autonomy
Evolution
Bias & Ethics
Side Effects and Reward Hacking
Transparency, Interpretability, and Explainability
Safety and AI

Machine Learning (ML) Overview

Forms of ML
ML Workflow
Selecting a Form of ML
Factors Involved in ML Algorithm Selection
Overfitting and Underfitting

ML Data

Data Preparation as Part of the ML Workflow
Training, Validation, and Test Datasets in the ML Workflow
Dataset Quality Issues
Data Quality and its Effect on the ML Model
Data Labelling for Supervised Learning

ML Functional Performance Metrics

Confusion Matrix
Additional ML Functional Performance Metrics for Classification, Regression, and Clustering
Limitations of ML Functional Performance Metrics
Selecting ML Functional Performance Metrics
Benchmark Suites for ML

ML Neural Networks and Testing

Neural Networks
Coverage Measures for Neural Networks

Testing AI-Based Systems Overview

Specification of AI-Based Systems
Test Levels for AI-Based Systems
Test Data for Testing AI-Based Systems
Testing for Automation Bias in AI-Based Systems
Documenting an AI Component
Testing for Concept Drift
Selecting a Test Approach for an ML System

Testing AI-Specific Quality Characteristics

Challenges Testing Self-Learning Systems
Testing Autonomous AI-Based Systems
Testing for Algorithmic, Sample, and Inappropriate Bias
Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems
Challenges Testing Complex AI-Based Systems
Testing the Transparency, Interpretability, and Explainability of AI-Based Systems
Test Oracles for AI-Based Systems
Test Objectives and Acceptance Criteria

Methods and Techniques for the Testing of AI-Based Systems

Adversarial Attacks and Data Poisoning
Pairwise Testing
Back-to-Back Testing
A/B Testing
Metamorphic Testing
Experience-Based Testing of AI-Based Systems
Selecting Test Techniques for AI-Based Systems

Test Environments for AI-Based Systems

Test Environments for AI-Based Systems
Virtual Test Environments for Testing AI-Based Systems

Using AI for Testing

AI Technologies for Testing
Using AI to Analyse Reported Defects
Using AI for Test Case Generation
Using AI for the Optimization of Regression Test Suites
Using AI for Defect Prediction
Using AI for Testing User Interfaces

How certification is earned

The course and syllabus include a one-hour multiple-choice exam. To earn certification, participants must achieve a score of 65% or higher. This e-proctored exam is scheduled on a separate day, not within the 4-day course. Additional exam time may be granted to eligible individuals under certain conditions to be agreed with iSQI.

Related Certifications

After completing this course attendees may consider:

ARTiBA Certified AI Engineering for Business and Management
ISTQB Certified Tester Advanced Level Test Analyst
ISTQB Certified Tester Advanced Level Technical Test Analyst
ISEB FoundationISQTB FoundationISTQBISTQB FoundationSoftware TestSoftware TestingAI TestingCertified AI Tester