AI evaluation, monitoring and change procedure

AI evaluation, monitoring and change procedure

AE-01 — Evaluation policy

[Organisation] evaluates AI capabilities against their intended use, requirements and risk context before relying on them within the approved scope. Evaluation considers the integrated system, including model, information preparation, prompts, retrieval, tools, interfaces and oversight. The evaluation owner records what the evidence can and cannot establish.

Criteria cover the properties required for the use, including functional correctness, validity and reliability, security, resilience, privacy, transparency and consequential effects on people. The owner selects measures appropriate to those properties and records the rationale. A single performance score is not used to stand in for all required properties.

AE-02 — Evaluation design

The owner defines the candidate configuration, intended users, operating conditions, task population and failure scenarios. Test information represents relevant conditions and identifies important limitations in coverage. The plan states expected outcomes, scoring or judgement methods, thresholds, reviewers, repeat conditions and evidence. Criteria are approved before the candidate’s result is judged.

Where outputs vary, the plan records sampling and repeated-run arrangements sufficient for the intended claim. It identifies uncertainty, sensitivity to configuration and the possibility that development has been tuned to the evaluation set. Human evaluation records the instructions, competence and disagreement handling used. Automated evaluators are themselves assessed for the limits of their judgements.

AE-03 — Technical and adversarial evaluation

The team tests relevant failure and misuse scenarios from the threat and risk assessments. These include incorrect or unsupported outputs, instruction manipulation, unintended disclosure, misuse of tools, excessive resource consumption and behaviour outside intended authority where applicable. Tests examine whether surrounding system controls prevent model outputs from acquiring permissions they do not hold.

Evaluation also examines ordinary integration behaviour: malformed output, unavailable services, timeout, retry, state change, incompatible versions and recovery. Privacy testing addresses relevant information exposure and processing requirements. Results distinguish a model refusing a request in one interaction from a system enforcing an authority or information boundary under the tested conditions.

AE-04 — Stakeholder and use validation

The validation owner assesses whether the capability supports the intended task under realistic conditions. The review considers how users understand limitations, how they rely on outputs, whether oversight is workable and whether adverse outcomes can be recognised and corrected. A warning in documentation is assessed for whether it changes actual use in the way assumed by the risk decision.

Where consequences may differ across relevant groups, languages or conditions, the evaluation identifies those differences and their uncertainty. The owner records whose experience is represented and where evidence is missing. Stakeholder effects are connected to requirements and decisions; they are not discarded because they are difficult to express as a single numerical measure.

AE-05 — Decision and reporting

The report identifies the candidate, criteria, methods, information used, results, uncertainty, failures and limitations. It preserves material individual failures alongside aggregate results where those failures affect the decision. The authority receives the report with risk treatment options and the operating conditions under which evidence supports use.

The decision records permitted use, limitations, required oversight, monitoring and reassessment. A bounded evaluation or pilot is identified as such. Changes after evaluation are assessed for whether the evidence still applies. The organisation does not report a supplier’s broad model score as a measured result for its own configured application.

AE-06 — Operational monitoring

The operations owner monitors the conditions and outcomes identified in the plan. Signals may include task failure, user correction, unusual tool use, information exposure, performance changes, service reliability, complaints and changes in input or output distributions. The record states collection purpose, privacy handling, review owner and response criteria.

Drift is assessed for its effect on the intended use rather than treated as automatically harmful or harmless. The owner compares observations with the approved operating basis and initiates investigation or reassessment when the recorded trigger is met. Monitoring gaps and delayed ground truth are identified so that confidence is not overstated.

AE-07 — Change, intervention and retirement

Changes to models, providers, prompts, retrieval, datasets, tools, permissions or populations trigger the recorded impact review. The owner determines which evaluation must be repeated and which earlier evidence remains applicable. The change record preserves the previous operating configuration and the reason for the new decision.

Intervention follows the authority and operating plan, which identifies how to restrict, suspend, replace or recover the capability. The owner verifies that the fallback functions as intended and records its limitations. Incidents, consequential failures and repeated deviations feed into the risk and improvement processes. Retirement closes access, monitoring and data obligations while retaining required decision evidence.

AI evaluation record: [AE ID; candidate; integrated configuration]
Intended use and population: [Tasks; users; conditions; stakeholder needs]
Criteria and method: [Properties; thresholds; sampling; reviewers]
Data and provenance: [Evaluation sets; sources; limitations; leakage checks]
Results: [Measures; consequential failures; variability; evidence]
Decision: [Authority; permitted scope; conditions; residual risk]
Monitoring: [Signals; criteria; owner; response; privacy handling]
Reassessment: [Triggers; changed elements; repeat evaluation; closure]