2 Framing and scoping the evaluation

2.1 One room, many expectations

2.2 Defining evaluation goals

2.2.1 Setting expectations

2.2.2 Comparing

2.2.3 Deciding go/no-go

2.2.4 Assessing risk

2.2.5 Ensuring compliance

2.2.6 Diagnosing and debugging

2.2.7 Negotiating and prioritizing evaluation goals

2.3 Determining the objects of the evaluation

2.4 Defining the task to be evaluated

2.4.1 Problems with AI task definitions

2.4.2 Defining an AI task well

2.5 Choosing evaluation dimensions

2.6 References

2.7 Summary

3 Picking evaluation metrics

3.1 “How do we measure this?”

3.2 What is a metric

3.3 What makes a good evaluation metric

3.3.1 Construct validity

3.3.2 Causal proximity

3.3.3 Movability

3.3.4 Reliability

3.3.5 Robustness

3.3.6 Correlation with human judgments

3.3.7 Interpretability and explainability

3.3.8 Measurability

3.4 Designing custom metrics

3.5 Managing and maintaining metrics

3.6 References

3.7 Summary

Overview

2 Framing and scoping the evaluation

Before evaluating an AI system, the chapter argues, you need to decide what you are trying to learn and what decisions the evaluation should support. It shows how teams can get lost when they jump straight to metrics without first aligning on goals, assumptions, and stakeholder needs. Using a medical advice chatbot as an example, it emphasizes that evaluation should be framed as a deliberate process of clarifying success, choosing priorities, and translating a vague question into a focused plan.

The chapter then outlines several common evaluation goals: setting realistic expectations, comparing systems or versions, making go/no-go deployment decisions, assessing risk, checking compliance, and diagnosing failures. These goals are distinct, often in tension, and should be prioritized based on decision urgency, lifecycle stage, domain risk, available resources, and desired scope and depth. The key message is that not every evaluation needs to do everything; in high-stakes settings like healthcare, safety, compliance, and risk often deserve attention before usability or general performance comparisons.

It also explains that a good evaluation requires choosing the right object, task, and dimensions to assess. For complex systems such as retrieval-augmented chatbots, this may mean evaluating the full system, a subsystem like retrieval or generation, or the underlying knowledge source depending on the goal. The chapter stresses the need to define the task precisely, including inputs, outputs, ground truth, exclusions, risk level, and autonomy, because ambiguity leads to scope mismatches and misleading judgments. Finally, it recommends selecting only the evaluation dimensions that inform the current decision, such as effectiveness, robustness, fairness, safety, explainability, security, efficiency, and broader societal impact.

Common goals of AI evaluation and the decisions they support.

Typical architecture of a retrieval-augmented generation (RAG) system. The system retrieves relevant content from an external knowledge base based on the user query and supplies it as context to a language model, enabling responses that combine domain knowledge with natural-language generation.

Two common sources of task-definition failure. In a scope mismatch, the system is expected to perform tasks it was never designed to handle while in an equivalence mismatch, stakeholders assume that solving one task is equivalent to solving a different one.

A process for transforming a vague task description into a precise and evaluable task definition. Each step narrows the space for interpretation and reduces the risk of misalignment between stakeholders, developers, and evaluators.

Risk-based classification of AI systems under the EU AI Act. The framework distinguishes between unacceptable, high, limited, and minimal risk systems, with regulatory requirements scaled according to the potential harm and impact of the system.

Common evaluation dimensions for AI systems. Beyond effectiveness, AI systems may need to be evaluated for properties such as robustness, fairness, safety, explainability, security, efficiency, and others.

Summary

Always start an evaluation by identifying what you need to learn from it and which decisions its results should support.
Prioritize your evaluation goals. Because different goals require different kinds of evidence and resources, focus on those that matter most given your current priorities, resources, and risks.
Make sure you have defined the AI task to be evaluated as precisely as possible. Specify the expected inputs and outputs, what counts as correct or acceptable behavior, what falls outside the task's scope, and its potential risks.
Select and focus on the aspects of system performance and behavior that matter for your goals, rather than attempting to measure everything.

FAQ

Why is framing and scoping an AI evaluation important before choosing metrics?

Framing and scoping come first because they clarify what you want to learn, what decision the evaluation must support, and what “success” means. Without that shared direction, teams often jump to metrics or benchmarks that do not answer the real question.

What are the main goals of AI evaluation described in this chapter?

The chapter highlights several common goals: setting expectations, comparing systems, deciding go/no-go for deployment, assessing risk, ensuring compliance, and diagnosing or debugging problems. Each goal supports a different kind of decision.

How do I know whether my evaluation should focus on expectations, comparison, or go/no-go?

Choose based on the decision you need to make. Use expectation-setting to align stakeholders on capabilities and limitations, comparison to judge a system against a baseline or alternative, and go/no-go when you need a clear deployment decision based on explicit thresholds.

When should I prioritize risk assessment in an AI evaluation?

Risk assessment should be a priority in safety- and trust-critical settings, especially before deployment, during audits, or when the system environment changes. It is especially important when failures could cause harm, even if average performance looks good.

What is the difference between compliance evaluation and risk evaluation?

Compliance evaluation checks whether the system meets legal, regulatory, and organizational requirements. Risk evaluation looks for harmful behaviors, vulnerabilities, and failure modes that may not be captured by formal requirements alone.

Why is diagnosing and debugging treated as an evaluation goal?

Because evaluation is not only about proving quality; it is also about finding root causes when problems occur. Diagnostic evaluation helps determine whether issues come from the model, data, retrieval, upstream components, or specific user segments.

What should I evaluate in a multi-component AI system like a RAG chatbot?

You should decide whether to evaluate the end-to-end system, the retrieval module, the generation module, the external knowledge base, or some combination. The right object depends on the goal: system-level behavior for expectations or compliance, and component-level analysis for comparison or debugging.

Why is defining the task so important before evaluation?

Because the task definition determines what counts as success, which metrics are relevant, what data to use, and how results should be interpreted. If the task is vague or mismatched, the evaluation can become misleading or unfair.

What are scope mismatch and equivalence mismatch?

Scope mismatch happens when the system is expected to do more than it was designed to do. Equivalence mismatch happens when two related but different tasks are treated as if they were the same, such as health education versus medical triage.

How should I choose which evaluation dimensions to include?

Start with the smallest set of dimensions that supports the immediate decision. Effectiveness is usually the baseline, while dimensions like robustness, fairness, safety, explainability, security, and efficiency should be added when they are relevant to the system’s context, risk level, and intended use.

2.2.1 Setting expectations

2.2.2 Comparing

2.2.3 Deciding go/no-go

2.2.4 Assessing risk

2.2.5 Ensuring compliance

2.2.6 Diagnosing and debugging

2.2.7 Negotiating and prioritizing evaluation goals

2.4.1 Problems with AI task definitions

2.4.2 Defining an AI task well

3.3.1 Construct validity

3.3.2 Causal proximity

3.3.3 Movability

3.3.4 Reliability

3.3.5 Robustness

3.3.6 Correlation with human judgments

3.3.7 Interpretability and explainability

3.3.8 Measurability

Common goals of AI evaluation and the decisions they support.

Typical architecture of a retrieval-augmented generation (RAG) system. The system retrieves relevant content from an external knowledge base based on the user query and supplies it as context to a language model, enabling responses that combine domain knowledge with natural-language generation.

Two common sources of task-definition failure. In a scope mismatch, the system is expected to perform tasks it was never designed to handle while in an equivalence mismatch, stakeholders assume that solving one task is equivalent to solving a different one.

A process for transforming a vague task description into a precise and evaluable task definition. Each step narrows the space for interpretation and reduces the risk of misalignment between stakeholders, developers, and evaluators.

Risk-based classification of AI systems under the EU AI Act. The framework distinguishes between unacceptable, high, limited, and minimal risk systems, with regulatory requirements scaled according to the potential harm and impact of the system.

Common evaluation dimensions for AI systems. Beyond effectiveness, AI systems may need to be evaluated for properties such as robustness, fairness, safety, explainability, security, efficiency, and others.

pro $24.99 per month

lite $19.99 per month

team

pro

team

pro

team