AI Model Evaluation

About this course

Design model evaluations that reflect real decisions, uncertainty, subgroup performance and changes after deployment.

This advanced course is part of the AI & Generative AI school. Work through the curriculum below in a instructor-led / cohort format, with an estimated study time of 28 h.

Course curriculum

• Module 1 — What Evaluation Is For
• Module 2 — Metrics and What They Hide
• Module 3 — Uncertainty and Fair Comparisons
• Module 4 — Calibration and Decision-Focused Evaluation
• Module 5 — Generalisation, Subgroups and Shift
• Module 6 — Evaluating Generative Systems
• Module 7 — Evaluation Integrity
• Module 8 — Evaluation in Operation and Reporting

Applied learning

• Case study — Tidewater Water: choosing, proving and running a failure-prediction model
• Lab 1 — Vendor A or vendor B?
• Lab 2 — External validation at site F
• Lab 3 — Audit an evaluation
• Capstone — An Evaluation Report

Who this course is for

Technology Professional; Data Professional; Engineer / Architect

Prerequisites

NE-AI-201

Course details

• Course code: NE-AI-306
• Level: Advanced
• Estimated duration: 28 h
• Delivery: Instructor-Led / Cohort