Home / Courses / VVUQ / Validation Metrics

VVUQ · Module 5 of 10

Validation Metrics and Validation Uncertainty

A comparison error means nothing without knowing how precisely it is known. The ASME V&V 20 method combines the numerical, input, and experimental uncertainties into one validation uncertainty, the yardstick the model error is measured against.

01

Readiness check

This module quantifies the validation. Tick only what you can do closed-notes.

  • Recall the comparison error E = S − D.
  • Recall the grid convergence index as a numerical uncertainty.
  • Combine independent uncertainties in quadrature.
  • Recall that a measurement carries an uncertainty.
  • Compare a value to an interval.
0 or 1 weak itemsContinue with this module.
2 weak itemsRevisit the GCI in Module 3.
3 or more weak itemsRevisit combining uncertainties in Mathematics, Module 15.
02

The core idea

The ASME V&V 20 validation uncertainty combines the numerical, input, and experimental uncertainties in quadrature. The comparison error E is compared to this uval: if |E| is smaller, the model error is hidden within the noise; if larger, a model discrepancy of about E is estimated.

E = S − Duval = √(unum2 + uinput2 + uD2)model error δmodel ≈ E ± uval

A comparison error is only interpretable alongside its uncertainty. The ASME V&V 20 standard defines a validation metric that makes this precise. The comparison error E = S − D estimates the model-form error, but it is contaminated by three known uncertainties: the numerical uncertainty unum from solution verification (the GCI), the input uncertainty uinput from imperfectly known simulation inputs, and the experimental uncertainty uD from the measurement. Assuming these are independent, they combine in quadrature into the validation uncertainty uval = √(unum2 + uinput2 + uD2). The model-form error is then estimated as δmodel ≈ E, known to within ±uval. The comparison is decisive: when |E| is smaller than uval, the model error cannot be distinguished from the combined noise, so the model is validated at that level; when |E| exceeds uval, a genuine model discrepancy of order E is detected. The validation uncertainty, not the raw error, is the standard against which a model is judged.

The skill works when: you combine the three uncertainties and compare the comparison error to uval.
The skill breaks down when: uncertainties are added linearly instead of in quadrature, or one source is left out.
The concept. Numerical, input, and experimental uncertainties combine in quadrature into the validation uncertainty. The comparison error is then measured against this single band.
03

The skills, taught in order

Five skills build the validation uncertainty and the metric that uses it.

5.1 The three uncertainty sources

A validation comparison carries three quantified uncertainties: numerical (from the GCI), input (from uncertain simulation inputs), and experimental (from the measurement). Each must be estimated in the same units as the compared quantity before they can be combined.

5.2 Combining in quadrature

Assuming the three sources are independent, they combine as the root sum of squares: uval = √(unum2 + uinput2 + uD2). Quadrature, not linear addition, is used because independent random errors partly cancel.

SymbolSourceFrom
unumnumerical errorsolution verification (GCI)
uinputuncertain inputspropagation of input uncertainty
uDexperimental errormeasurement uncertainty
uvalvalidation uncertaintyquadrature of the three

The ASME V&V 20 validation uncertainty gathers every known error source into one band against which the model is judged.

5.3 The validation comparison

The model-form error is estimated as δmodel ≈ E, with an uncertainty of ±uval. When |E| ≤ uval, the model error is within the noise and the model is validated at that uncertainty; when |E| > uval, a discrepancy of about E is detected.

5.4 Interpreting the result

A validated result does not mean zero error; it means the model error is no larger than uval. Reducing uval, by a finer mesh, better inputs, or better measurements, sharpens the test and can reveal a discrepancy that a loose validation hid.

5.5 Validation metrics

The comparison error with its validation uncertainty is one validation metric. Others compare whole distributions, such as the area between the simulated and measured cumulative distributions, capturing more than a single point. The ASME multivariate metrics extend this to several outputs at once.

Engineering connection: reporting a validation as E ± uval is what lets a reviewer decide whether a model is accurate enough for a specific decision.

04

Worked example 1: the validation uncertainty

A validation has a numerical uncertainty unum = 0.02 (from the GCI), an input uncertainty uinput = 0.03, and an experimental uncertainty uD = 0.04. Find the validation uncertainty.

Figure 1. The three uncertainties combine in quadrature, like the legs of a right triangle in higher dimensions. The result, 0.054, is dominated by the largest source.
  1. ProblemFind the validation uncertainty for the sources in Figure 1.
  2. Given / findunum = 0.02, uinput = 0.03, uD = 0.04. Find uval.
  3. AssumptionsThe three sources are independent, so they combine in quadrature.
  4. Modeluval = √(unum2 + uinput2 + uD2).
  5. Equationsuval = √(0.022 + 0.032 + 0.042)
  6. SolveSum of squares = 0.0004 + 0.0009 + 0.0016 = 0.0029. uval = √0.0029 = 0.054.
  7. CheckThe result exceeds the largest single source (0.04) but is far below their linear sum (0.09), the signature of quadrature. The experimental term dominates.
  8. ConclusionThe validation uncertainty is 0.054. Any comparison error must be judged against this band, not against zero.
Result. uval = 0.054.
05

Worked example 2: is the model validated?

For that validation (uval = 0.054), the comparison error is E = 0.03. Decide whether the model is validated at this uncertainty level.

Figure 2. The comparison error falls inside the validation uncertainty band, so the model error cannot be distinguished from the combined numerical, input, and experimental noise.
  1. ProblemJudge whether the model is validated for the case in Figure 2.
  2. Given / findE = 0.03, uval = 0.054. Compare |E| to uval.
  3. AssumptionsThe three uncertainties were correctly estimated and combined.
  4. ModelIf |E| ≤ uval, the model error is within the noise; otherwise a discrepancy of about E is detected.
  5. Equationscompare |E| = 0.03 to uval = 0.054
  6. Solve|E| = 0.03 < uval = 0.054, so the comparison error is within the validation uncertainty: the model error cannot be distinguished from noise, and the model is validated at the 0.054 level.
  7. CheckValidation here means the model error is no larger than 0.054, not that it is zero. A tighter uval, from better data or mesh, could still expose a discrepancy.
  8. ConclusionThe model is validated at this uncertainty: its error is indistinguishable from the combined noise. Sharpening the uncertainties would strengthen the test.
Result. |E| = 0.03 < uval = 0.054: validated at this uncertainty level.
06

Misconceptions and diagnostics

MistakeSymptomDiagnostic questionCorrection
Adding uncertainties linearlyuval too large"Did I use quadrature?"Combine independent sources as a root sum of squares.
Omitting a sourceuval too small"Did I include numerical, input, and data?"All three enter the validation uncertainty.
Validated means exactZero error assumed"What does validated mean here?"It means the error is no larger than uval.
Judging E against zeroSmall error called a discrepancy"Compared to what band?"Judge E against uval, not against zero.
07

Practice ladder

Level 1 · Direct skill

Combine unum = 0.03 and uD = 0.04 in quadrature (no input uncertainty).

Show answer

uval = √(0.032 + 0.042) = √0.0025 = 0.05.

Level 2 · Mixed concept

If the comparison error there is E = 0.08, is the model validated at that level?

Show answer

|E| = 0.08 > uval = 0.05, so a real model discrepancy of about 0.08 is detected: not validated at this level.

Level 3 · Independent problem

Uncertainties are unum = 0.01, uinput = 0.02, uD = 0.02, and E = 0.015. Validated?

Show answer

uval = √(0.0001 + 0.0004 + 0.0004) = √0.0009 = 0.03. |E| = 0.015 < 0.03, so validated at the 0.03 level.

Transfer task | Real engineering

Your validation passes, but the experimental uncertainty is large. Explain why that is a weak result and how to strengthen it.

What good work looks like

A large uD inflates uval, so the model passes only because the test is imprecise; a hidden discrepancy could be masked. Reducing the measurement uncertainty tightens uval and makes the validation a stronger, more discriminating test.

08

Working with AI, and proving it yourself

Use AI as an examiner, not a solver

"Check that I combined the three uncertainties in quadrature, not linearly."
"Give me three validation cases; I will decide if each is validated."
"Is my model validated?" Computing uval and comparing yourself is the skill.
"What is the model error?" Reasoning from E and uval is the point.

Portfolio task

For a real validation, estimate the numerical, input, and experimental uncertainties, combine them into uval, and report the result as E ± uval with a validated-or-not verdict.

Must include: three uncertainty estimates, a uval, and a comparison to E.
09

Retrieval and spaced review

Closed notes. Answer out loud, then reveal.

1. Name the three uncertainty sources in a validation.

Numerical, input, and experimental.

2. Write the validation uncertainty.

uval = √(unum2 + uinput2 + uD2).

3. When is a model validated at a level?

When |E| is within uval, so the error is indistinguishable from noise.

4. Does validated mean exact?

No: it means the model error is no larger than uval.

5. How is the test made stronger?

By reducing uval through a finer mesh, better inputs, or better measurements.

TodayFinish this quiz and Levels 1 and 2 of the ladder.
+1 dayRe-derive uval and the verdict from a blank page.
+3 daysJudge three new validation cases.
+7 daysTurn to the uncertainties themselves, Module 6.
+30 daysReuse E ± uval when reporting any validation.
10

Standards mapping

This module follows the ASME Verification, Validation, and Uncertainty Quantification standards. Use these references to read further.

Topic in this moduleWhere to read more
Validation uncertainty and comparison errorASME V&V 20, CFD and Heat Transfer
Multivariate validation metricsASME VVUQ 20.1, Multivariate Metrics
Role of uncertainty quantificationASME VVUQ 10.2, UQ in Solid Mechanics

Standard designations refer to the ASME V&V series; the validation-uncertainty method is the core of ASME V&V 20.