VVUQ · Module 1 of 10
The VVUQ Framework and Model Credibility
Three questions decide whether a simulation can be trusted: are the equations solved right, does the model match reality, and how big is the uncertainty. Verification, validation, and UQ answer them in that order.
Readiness check
Learning objectives
- Distinguish verification, validation, and uncertainty quantification by the question each answers.
- Calculate the comparison error E = S − D and interpret its sign.
- Justify why verification must precede validation.
- Identify what sets the required model credibility for a stated intended use.
This module opens the course. Tick only what you can do closed-notes.
- Compute a relative error between two numbers.
- Recall that a simulation approximates a mathematical model of reality.
- Distinguish a coding error from a modeling assumption.
- Recall that measurements carry uncertainty.
- State what decision a model is meant to support.
The core idea
Verification asks whether the equations are solved correctly; validation asks whether the right equations were chosen; uncertainty quantification asks how much the answer could be off. The gap between simulation and reality mixes all three, so they must be separated.
verification: solving the equations rightvalidation: solving the right equationscomparison error E = S − DA computational result is a prediction that must earn trust before it carries a decision. The ASME VVUQ framework separates the ways it can be wrong. Verification is a mathematics question: does the code solve the chosen equations correctly (code verification), and is the particular solution converged enough (solution verification). Validation is a physics question: do those equations represent reality, judged by comparing the simulation S to experimental data D. Uncertainty quantification asks how much S and D could vary given imperfect inputs, models, and measurements. The observed comparison error E = S − D is not pure model error: it blends numerical error from verification, input uncertainty, and experimental uncertainty. That is why the order matters. You verify first, so that a validation disagreement is not blamed on the physics when it is really a coarse mesh. Credibility, in the ASME V&V 40 sense, is then the accumulated evidence that the model is adequate for its intended use, matched to the consequence of the decision it supports.
The skills, taught in order
Five skills fix the vocabulary and the process that the rest of the course builds on.
1.1 Verification
Verification is purely mathematical: it checks that the equations are solved correctly, with no reference to reality. Code verification provides evidence that the implemented algorithms behave as intended for the equations, terms, boundary conditions, and options the tests exercise; it does not establish that a software product is free of defects. Solution verification estimates the numerical uncertainty of the specific run. Establish verification evidence before you interpret a validation gap, or numerical error will be mistaken for physics.
1.2 Validation
Validation checks that the equations represent the real world, by comparing the simulation to experimental data over the intended range of use. It can only be as good as the data and the verification behind it; a validation claim on an unverified solution is meaningless.
1.3 Uncertainty quantification
UQ characterises how variations in inputs, models, and measurements affect the result. It turns a single number into a number with an interval, which is what a decision actually needs. Verification and validation errors are among the uncertainties it accounts for.
| Activity | Question | Compared against |
|---|---|---|
| Verification | solving the equations right? | the mathematics |
| Validation | solving the right equations? | experimental data |
| Uncertainty quantification | how far could it be off? | input and model variation |
The three VVUQ activities and the distinct question each answers. Confusing them is the most common mistake in the field.
1.4 The comparison error
The comparison error E = S − D is the raw disagreement between simulation and experiment. It is a mix of numerical error, model-form error, input uncertainty, and experimental uncertainty. Building an uncertainty budget for E, asking how much of the gap each source could account for rather than blaming the model, is the analytical heart of validation.
1.5 Model credibility and intended use
Credibility is the body of VVUQ evidence that a model is adequate for a specific intended use. The ASME V&V 40 approach scales the required rigor to the model's influence on a decision and the consequence of that decision being wrong: high-stakes uses demand more evidence.
Engineering connection: a stress or flow simulation cannot support a certification decision until it carries verification, validation, and uncertainty evidence proportional to what failure would cost.
Worked example 1: the comparison error
A simulation predicts a peak stress of S = 1.05 (normalised), while a validation experiment measures D = 1.00. Find the comparison error and the relative disagreement.
- ProblemFind the comparison error and relative disagreement in Figure 1.
- Given / findS = 1.05, D = 1.00. Find E and the relative error.
- AssumptionsThe solution is verified, so E is a meaningful physics comparison, not a mesh artefact.
- ModelE = S − D; relative error = E/D.
- EquationsE = S − Drelative = E/D
- SolveE = 1.05 − 1.00 = 0.05. Relative = 0.05/1.00 = 5%.
- CheckThe 5% gap is the total disagreement; whether it signals a real model error depends on the experimental and numerical uncertainties, examined in later modules.
- ConclusionThe simulation is 5% above the experiment. That number opens the validation question but does not settle it, because the gap has several sources.
Worked example 2: reading the gap
For that 5% comparison error (E = 0.05), a grid convergence study estimates the numerical standard uncertainty as unum = 0.012. Decide whether the numerical contribution is large enough to account for the gap on its own, and say what that does and does not establish.
- ProblemDecide whether the numerical contribution in Figure 2 can account for the gap on its own.
- Given / findE = 0.05, unum = 0.012. Compare the numerical-uncertainty estimate with the observed discrepancy magnitude.
- AssumptionsThe grid study's uncertainty estimate is valid for this run. Input and experimental uncertainty are not quantified here, so they are unknown rather than zero.
- ModelE contains model-form, numerical, input, and experimental contributions. Their signs are generally unknown and they can partly cancel, so E is compared against each estimate rather than split by subtraction.
- EquationsE = S − D = 0.05unum/|E| = 0.012/0.05 = 0.24
- SolveUnum/|E| = 0.012/0.050 = 0.24. The numerical-uncertainty estimate is about a quarter of the observed discrepancy magnitude, so numerical uncertainty alone is unlikely to account for it.
- CheckRefining the mesh addresses only the numerical contribution, so it is unlikely to close this discrepancy on its own. That is the useful conclusion. What the comparison does not do is partition the discrepancy: subtracting an uncertainty estimate from a signed discrepancy is not a defined operation, input and experimental uncertainty have not been quantified, and contributions can cancel, so a small discrepancy would not prove a small model-form contribution either.
- ConclusionVerifying first showed that a finer mesh will not close this gap. The next step is to quantify input and experimental uncertainty, not to revise the physics on the strength of a subtraction.
Misconceptions and diagnostics
| Mistake | Symptom | Diagnostic question | Correction |
|---|---|---|---|
| Confusing verification and validation | Comparing to data to check the math | "Am I checking the equations or the physics?" | Verification uses mathematics; validation uses experiments. |
| Validating an unverified solution | Blaming physics for a mesh error | "Is the solution converged?" | Verify first, then validate. |
| Reporting a bare number | A prediction with no interval | "What is the uncertainty?" | Attach a quantified uncertainty to every result. |
| Ignoring intended use | Same rigor for every model | "What decision does this support?" | Scale credibility to the consequence. |
Practice ladder
A model predicts 240 N and the test measures 250 N. Find the comparison error and relative error.
Show answer
E = 240 − 250 = −10 N; relative = −10/250 = −4%.
Classify each as verification, validation, or UQ: (a) refining the mesh, (b) comparing to a wind-tunnel test, (c) sampling uncertain material properties.
Show answer
(a) verification (solution), (b) validation, (c) uncertainty quantification.
A 6% comparison error has an estimated numerical uncertainty of 0.01 (relative). What can you conclude, and what can you not?
Show answer
Unum/|E| = 0.01/0.06 = 0.17, so the numerical-uncertainty estimate is about one sixth of the observed discrepancy magnitude and is unlikely to account for it alone. You cannot conclude anything about the size of the model-form contribution: the discrepancy is not partitioned by this comparison, input and experimental uncertainty are unquantified, and contributions can partly cancel.
For a simulation that will support a safety certification, outline the verification, validation, and UQ evidence you would need before trusting it.
What good work looks like
Code and solution verification (order of accuracy, a grid study), validation against relevant experiments with quantified data uncertainty, and propagated input and model uncertainty, with the rigor scaled to the certification consequence per the credibility framework.
Working with AI, and proving it yourself
Use AI as an examiner, not a solver
Portfolio task
Take a simulation you have run, state the decision it supports, and lay out the verification, validation, and UQ evidence it currently has and lacks.
Retrieval and spaced review
Closed notes. Answer out loud, then reveal.
1. What does verification check?
That the equations are solved correctly, a purely mathematical question.
2. What does validation check?
That the equations represent reality, by comparison with experiment.
3. Why verify before validating?
So a validation gap is not blamed on the physics when it is numerical error.
4. Write the comparison error.
E = S − D, simulation minus experimental data.
5. What sets the required credibility?
The model's influence on a decision and the consequence of being wrong.