VVUQ · Module 4 of 10
Validation Experiments and the Validation Hierarchy
Validation compares a simulation to reality, but only over the conditions actually tested. A validation hierarchy builds trust from simple pieces up to the full system, and predictions are safe only inside the validated range.
Readiness check
This module compares models to experiments. Tick only what you can do closed-notes.
- Recall the comparison error E = S − D.
- Recall that a measurement has an uncertainty.
- Distinguish interpolation from extrapolation.
- Recall that a model is validated only where tested.
- Recall a dimensionless group such as the Reynolds number.
The core idea
Validation compares a verified simulation to experimental data designed for the purpose. A validation hierarchy builds evidence from unit problems up to the full system, and a model is only credible for predictions inside the range where it was validated.
compare S to D at tested conditionshierarchy: unit → benchmark → subsystem → systempredict inside the validated range, not beyondOnce a solution is verified, validation asks whether the model matches reality. This is done with validation experiments, tests designed specifically to measure the quantities a simulation predicts, with well-characterised conditions and quantified measurement uncertainty, not repurposed qualification tests. Because a full engineering system is too complex to validate in one step, validation is built as a hierarchy: unit-level problems isolate single physics, benchmark cases combine a few, subsystem tests add coupling, and only at the top does the complete system appear. Evidence accumulates up the pyramid, so confidence in a system prediction rests on validated pieces beneath it. The comparison error E = S − D is judged against the experimental uncertainty: a gap smaller than the data uncertainty cannot be distinguished from measurement noise, while a larger gap signals a real model discrepancy. Crucially, validation certifies the model only over the conditions tested. Using it to predict within that range is interpolation, which the validation supports; using it beyond the range is extrapolation, which it does not, and which demands extra justification.
The skills, taught in order
Five skills build validation experiments, the hierarchy, and the limits of a validation.
4.1 Validation experiments
A validation experiment is designed to test a model, not to qualify a product: it measures the predicted quantities under controlled, well-characterised conditions with quantified uncertainty. The simulation should ideally be run before the data are seen, to avoid tuning the model to the answer.
4.2 Comparing simulation to data
The comparison error E = S − D is judged against the experimental uncertainty uD. If |E| is smaller than uD, the disagreement is within measurement noise; if larger, a real model discrepancy is indicated, of size roughly |E|.
| Level | What it tests | Physics |
|---|---|---|
| Unit problem | a single effect | isolated |
| Benchmark | a few effects | partly coupled |
| Subsystem | a component | coupled |
| System | the full application | fully coupled |
The validation hierarchy. Simpler levels isolate physics and are easier to measure precisely; the system level is the goal but the hardest to test.
4.3 The validation hierarchy
Because a full system cannot be validated in one experiment, evidence is built up a hierarchy from unit problems to the complete application. Each level validates the physics it adds, so a credible system prediction stands on a foundation of validated pieces.
4.4 The validation domain
A validation certifies the model only over the range of conditions actually tested, the validation domain. Predictions at conditions between tested points are interpolation, supported by the validation; predictions outside are extrapolation, which the validation does not cover.
4.5 Interpolation versus extrapolation
Using a validated model inside its domain carries the validated uncertainty. Using it outside requires additional justification, because the physics may change and no data constrain the model there. Recognising which side of the boundary a prediction falls on is essential to credibility.
Engineering connection: a turbulence model validated at one Reynolds number range cannot be assumed valid at another without new evidence, a distinction that governs every simulation-based decision.
Worked example 1: is the disagreement real?
A simulation predicts S = 1.05 and a validation experiment measures D = 1.00 with an experimental uncertainty uD = 0.03. Decide whether the comparison error signals a real model discrepancy.
- ProblemJudge whether the comparison error in Figure 1 is a real discrepancy.
- Given / findS = 1.05, D = 1.00, uD = 0.03. Compare |E| to uD.
- AssumptionsThe solution is verified, so E is a physics comparison; input uncertainty is small here.
- ModelE = S − D; a discrepancy is indicated when |E| exceeds the data uncertainty uD.
- EquationsE = S − Dcompare |E| to uD
- SolveE = 1.05 − 1.00 = 0.05. |E| = 0.05 > uD = 0.03, so the gap exceeds the measurement uncertainty: a real model discrepancy of about 0.05 is indicated.
- CheckThe simulation sits outside the experiment's ±0.03 band, so the disagreement cannot be explained by measurement noise alone, unlike a case where |E| < uD.
- ConclusionThe 0.05 gap is a genuine model discrepancy, not noise. Comparing the error to the data uncertainty is what separates the two.
Worked example 2: inside or outside the validation domain
A model is validated at Reynolds numbers 1000 (comparison error 2%) and 5000 (3%). Judge its credibility for predictions at Re = 3000 and at Re = 10000.
- ProblemJudge the two predictions against the validation domain in Figure 2.
- Given / findValidated at Re = 1000 (2%) and Re = 5000 (3%). Assess Re = 3000 and Re = 10000.
- AssumptionsThe physics does not change abruptly within the tested range; no data exist beyond it.
- ModelPredictions between tested points are interpolation (supported); beyond them, extrapolation (unsupported).
- Equationsvalidated domain: 1000 ≤ Re ≤ 5000
- SolveRe = 3000 lies between 1000 and 5000, so it is interpolation, credible with roughly the validated 2 to 3% error. Re = 10000 lies outside, so it is extrapolation, not covered by the validation and requiring new evidence.
- CheckThe validation certifies only 1000 to 5000; the flow regime could shift by Re = 10000, so the validated error cannot be assumed to hold there.
- ConclusionThe Re = 3000 prediction is defensible; the Re = 10000 prediction is not, without additional validation. The domain boundary decides.
Misconceptions and diagnostics
| Mistake | Symptom | Diagnostic question | Correction |
|---|---|---|---|
| Ignoring data uncertainty | Small gap called a discrepancy | "Is |E| bigger than uD?" | Compare the error to the measurement uncertainty. |
| Extrapolating a validation | Trusting the model out of range | "Is this inside the validated domain?" | Validation covers only the tested conditions. |
| Tuning to the data | Model adjusted after seeing D | "Was S predicted blind?" | Predict before comparing to avoid calibration-as-validation. |
| Validating only the system | No supporting unit evidence | "Are the lower levels validated?" | Build the hierarchy from unit problems up. |
Practice ladder
S = 98, D = 100, uD = 5. Is the disagreement within measurement noise?
Show answer
|E| = 2 < uD = 5, so yes: the gap is within the measurement uncertainty and cannot be distinguished from noise.
Order these validation levels from base to peak: subsystem, unit, system, benchmark.
Show answer
Unit, benchmark, subsystem, system, from isolated physics up to the full application.
A model is validated for loads 10 to 50 kN. Classify predictions at 30 kN and 70 kN.
Show answer
30 kN is inside the domain (interpolation, supported); 70 kN is outside (extrapolation, not covered without new validation).
You must trust a system-level simulation for a new operating point. Describe the validation evidence you would want beneath it.
What good work looks like
Validated unit and benchmark cases for each key physics, subsystem tests for the couplings, and system data bracketing the operating point, so the prediction is interpolation within a hierarchy of validated evidence, not an unsupported extrapolation.
Working with AI, and proving it yourself
Use AI as an examiner, not a solver
Portfolio task
For a real validation, plot simulation against data with the measurement uncertainty, state the validation domain, and classify an intended prediction as interpolation or extrapolation.
Retrieval and spaced review
Closed notes. Answer out loud, then reveal.
1. What is a validation experiment?
A test designed to measure the quantities a model predicts, with quantified uncertainty.
2. When is a comparison error a real discrepancy?
When |E| exceeds the experimental uncertainty uD.
3. What is the validation hierarchy?
Evidence built from unit problems up through subsystems to the full system.
4. What is the validation domain?
The range of conditions over which the model was tested.
5. Interpolation versus extrapolation?
Predicting inside the domain is supported; outside it is not, without new evidence.
Standards mapping
This module follows the ASME Verification, Validation, and Uncertainty Quantification standards. Use these references to read further.
| Topic in this module | Where to read more |
|---|---|
| Validation experiments and the hierarchy | ASME V&V 10, Computational Solid Mechanics |
| Comparing simulation to data | ASME V&V 20, CFD and Heat Transfer |
| Validation domain and prediction | Oberkampf and Roy, Verification and Validation in Scientific Computing |
Standard designations refer to the ASME V&V series; the validation hierarchy is developed in V&V 10 and the reference by Oberkampf and Roy.