Define the role of AI
AI may classify, estimate, predict, generate or recommend. The system boundary must distinguish model output from an engineering decision and identify who remains accountable.
- State the decision being assisted.
- Define unacceptable failure modes.
- Separate advisory output from automatic action.
- Provide a deterministic fallback or escalation path.
Control data and physical context
Training and evaluation data must represent the intended operating envelope. For physical systems, units, conservation behaviour, geometry, boundary conditions and configuration can be as important as sample volume.
- Trace data origin, transformation and permission.
- Separate training, calibration and evaluation sets.
- Test rare, boundary and out-of-distribution conditions.
- Use physics constraints only where their assumptions remain valid.
Evaluate beyond average accuracy
Aggregate accuracy can hide unsafe regional behaviour. Evaluation should examine error distribution, calibration, uncertainty, robustness, sensitivity and performance for relevant operating segments.
- Compare against simple and physics-based baselines.
- Report false-positive and false-negative consequences.
- Test perturbation, missing data and sensor drift.
- Define confidence thresholds tied to action authority.
Operate the model as a controlled component
Production AI needs versioning, observability, drift detection, incident handling and rollback. A changed model is a changed system, even when the user interface remains unchanged.
- Version model, data, features and configuration.
- Monitor inputs, outputs, errors and interventions.
- Require evidence before expanding the operating envelope.
- Retain human override and rollback capability.
Apply the method to the decision—not as a checklist.
The appropriate evidence depends on intended use, technical risk, operating environment, contractual obligations and the authority responsible for acceptance. This guide is educational and does not replace project-specific analysis, applicable standards or independent review.